X-Git-Url: http://plrg.eecs.uci.edu/git/?a=blobdiff_plain;f=utils%2FNightlyTestTemplate.html;h=c38bb2e776bbe4e7aadfa462cd8d06863c319c0b;hb=86737665b81550fdb575f7d8cc5decc801a7813d;hp=ed00481a910d9cf7689f9863ae5b65b0361d9f31;hpb=2f8cb571691b33cfe521e6d4185a04c5b8745d11;p=oota-llvm.git
diff --git a/utils/NightlyTestTemplate.html b/utils/NightlyTestTemplate.html
index ed00481a910..c38bb2e776b 100644
--- a/utils/NightlyTestTemplate.html
+++ b/utils/NightlyTestTemplate.html
@@ -1,4 +1,3 @@
-
LLVM Test Results for $DateString
@@ -12,12 +11,12 @@
@@ -30,8 +29,8 @@
Back to:
-Test Results
-LLVM Page
+Test Results
+LLVM Page
@@ -41,25 +40,44 @@
Today's Test Results Overview
|
+
+
+
+Nightly Test Overview:
+
+ - Start: $TestStartTime
+ - Finish: $TestFinishTime
+ - Platform: $TestPlatform
+
CVS Tree Overview:
- CVS Checkout Log
- Time to check out CVS tree: $CVSCheckoutTime seconds
- Number of directories in CVS: $NumDirsInCVS
- Number of files in CVS: $NumFilesInCVS
- Number of lines of code: $LOC
+ $NumDirsInCVS dirs, $NumFilesInCVS files, $LOC
+ lines of code, checked out in $CVSCheckoutTime seconds
Compilation Log
-
- Time to build CVS tree: $BuildTime seconds
- Number of object files compiled: $NumObjects
- Number of libraries linked: $NumLibraries
- Number of executables linked: $NumExecutables
-
+
+Item | CPU Time | Wall Clock |
+Configure CVS Tree | $ConfigTime | $ConfigWallTime |
+Build CVS Tree | $BuildTime | $BuildWallTime |
+Run Dejagnu Tests | $DejagnuTime | $DejagnuWallTime |
+
+Number of object files compiled: $NumObjects
+Number of libraries linked: $NumLibraries
+Number of executables linked: $NumExecutables
+Build Status: $BuildStatus
Warnings during the build:
-$WarningsList
+$WarningsList
@@ -77,34 +95,80 @@ $WarningsList
Changes to Warnings:
-
-- Warnings Added: $WarningsAdded
-
- Warnings Removed: $WarningsRemoved
-
+ Warnings Added:
+$WarningsAdded
+Warnings Removed:
+$WarningsRemoved
Changes in the test suite:
-- New Tests: ...
-
- Removed Tests: ...
-
- Newly passing tests: ...
-
- Newly failing tests: ...
+
- New Tests: $TestsAdded
+
- Removed Tests: $TestsRemoved
+
- Newly passing tests: $TestsFixed
+
- Newly failing tests: $TestsBroken
+ |
+
+
+
+
+
+$DejagnuTestResults
+A complete log of testing Feature and Regression is available for further analysis.
-GNUplot charts showing a couple of things:
-
-- LOC in CVS
-
- Compile Time
-
- Testing Time
-
- Time to run some programs, say all of Olden
-
- Time to compile something, say Olden.
-
- Size of bytecode files for Olden
-
+
+Here are some charts showing how the LLVM optimizer and code generators are
+changing over time. For now we use the Olden benchmark suite to measure this,
+but eventually we will switch to using SPEC CPU2000. All programs are run with
+"LARGE_PROBLEM_SIZE" enabled. Click on any of the charts to get a larger
+version.
+
+
Compilation Measurements:
+
+
+
+
+
+Size of LLVM bytecode files
+ |
+
+
+Time to run the LLVM optimizer on each program
+ |
+
+
+Program Execution Measurements:
+
+
+
+
+
+Execution time for CBE generated executable
+ |
+
+
+Execution time for the LLC generated executable
+ |
+
+
+
+
+Execution time for program in the JIT
+ |
+ |
+
+
+
+
@@ -112,16 +176,69 @@ GNUplot charts showing a couple of things:
Program Tests
|
-Not yet implemented
+This section tests LLVM on a variety of programs in the test suite. This
+includes benchmark suites like the Olden, McCat, Ptrdist, and SPEC benchmarks as
+well as a few random programs with test inputs. This section is meant to track
+how stable LLVM is as a whole. A failure in the execution of any test is marked
+with an asterisk: `*'. The columns of the tables are:
-
+
+- Program - The name of the program for that row.
+- GCCAS - Time to run LLVM optimizers on the program.
+- Bytecode - The size of the bytecode for the
+ program
+- Instrs - The number of LLVM instructions in the
+ compiled bytecode
+- LLC compile - The time taken compile with
+ LLC (the static backend)
+- JIT codegen - The amount of time spent in the
+ JIT itself, instead of executing the program.
+- Machine code - The number of bytes of machine
+ code generated by the JIT.
+- GCC - The time taken to execute the program when compiled
+ with GCC -O2.
+- CBE - The time taken to execute the program after
+ compilation through the C backend, compiled with -O2.
+- LLC - How long does the program generated by the static
+ backend LLC take to execute
+- JIT - The amount of time spent running the
+ program with the JIT; this includes the code generation phase (listed above)
+ and actually running the program.
+- GCC/LLC - The speed-up of the LLC output vs the native
+ GCC output: greater than 1 is a speedup, less than 1 is a slowdown.
+- GCC/CBE - The speed-up of the CBE output vs the native
+ GCC output: greater than 1 is a speedup, less than 1 is a slowdown.
+- LLC-BETA - How long does the program generated by the static
+ backend LLC take to execute the program, when compiled with new experimental
+ features. This is temporary, for tuning.
+
+
+A complete log of testing
+SingleSource,
+MultiSource, and
+External programs are
+available for further analysis.
+
+
Programs/External
+
+
+$ExternalProgramsTable
+
-Not yet implemented
+
Programs/MultiSource
-
+
+
+$MultiSourceProgramsTable
+ |
+
+Programs/SingleSource
+
+
+
+$SingleSourceProgramsTable
+ |
+
+
-