1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
|
.TH NUMBENCH "1" "March 2012" "Gentoo" "User Commands"
.SH NAME
numbench \- Benchmarking tool for Gentoo
.SH SYNOPSIS
.B numbench
module conffile [\fIoptions\fR]
.B numbench
module [\fI-h|--help\fR]
.SH DESCRIPTION
.PP
numbench is a tool for compiling and benchmarking numerical
libraries. It is useful to find out the implementations of common
libraries that run faster on the machine and are more accurate.
The script makes use of the portage features in order to download,
compile, install and test the packages.
.SS Modules
.IX Subsection "Modules"
.PP
The following modules are available:
.RS
.B blas
\- Test BLAS implementations
.B cblas
\- Test CBLAS implementations
.B lapack
\- Test LAPACK implementations
.B scalapack
\- Test the ScaLAPACK libary
.B blas_accuracy
\- Test BLAS implementations for accuracy
.B lapack_accuracy
\- Test LAPACK implementations for accuracy
.B fftw
\- Test the FFTW libary
.B metis
\- Test the METIS tools
.RE
.PP
More information about each module is available through the command:
.B numbench module --help
.SS Configuration file
.IX Subsection "Configuration file"
.PP
In order to run a test, a configuration file has to be provided.
The configuration file is an XML document: the main tag has the name
.I tests
and it contains one ore more test case definitions within tags
.I test
. Each test has to specify an id within the corresponding argument and a
package in a format understandable by Portage within the tag
.I pkg.
This package is supposed to contain (at least) an implementation that can
be tested by numbench.
.PP
Within each
.I test
tag up to four different environments can be tested. These are dependenv,
emergeenv, compileenv and runenv. In order to specify an environment,
create a tag with the same name, and add in this tag as many variable
definitions as you desire. To define a variable it sufficient to create
a tag
.I var
with the variable name in the argument "name" and the value as tag text.
.PP
The following example shows a minimal configuration file for blas
implementations, where the test "reference" is defined; the package
sci-libs/blas-reference-20120215-r1 is required and an emergeenv environment
with the variable FFLAGS equal to "-O3 -funroll-all-loops"
is defined:
<tests>
<test id="reference">
<pkg>sci-libs/blas-reference-20120215-r1</pkg>
<emergeenv>
<var name="FFLAGS">-O3 -funroll-all-loops</var>
</emergeenv>
</test>
</tests>
.PP
The "dependenv" environment is used while emerging the dependencies of the
package; the environment named "emergeenv" is used while emerging the package
itself; "compileenv" is used when compiling the test program (if any) and
"runenv" represents the environment within which the test program is run. The
environments can also been specified through external files: if the argument
"script" is given, its value must be the path of a script that exports the
desired environment variables; at the end of the execution the environment is
read and used. If the argument "file" is given, it must be the path of a text
file containing key:value pairs, one on each line. If the argument "append" is
given and its value is 1, the variables specified override the variables of
the environment numbench is running in and this resulting environment is
used.
.PP
If the package installs more than one implementation, one or more of them can
be skipped in order to reduce the required time. In order to skip one or more
implementations, many tags "skip" can be created within a single "test" tag.
The text value of these tags is intepreted as glob pattern. Regular expressions
can be used: in this case it sufficient to add the argument "type" with value
"regexp" to the "skip" tag.
.PP
An useful feature of numbench is the ability of changing the implementation of
a library used by a tested software. For example, most lapack implementations
internally use a blas implementation just like other libraries or programs. You
could decide to test the reference-lapack implementation with two different
underlying blas implementations. In order to do that it is sufficient to create
within the "test" tag a tag named "required", with an argument named "name"
whose value is the generic name of the library (e.g. "blas"), and a text
value corresponding to the specific name of the library to use (e.g. "eigen").
The required implementation must be installed in the main system.
.PP
Examples of configuration files are available under /usr/share/numbench/samples
and cover a wide range of possible test cases.
.SS Logs
.PP
A complete set of log is saved during the execution of the script. The script
itself will print at the beginning the directory where the logs are saved.
In the directory you will find the file main.log, which contains everything
that is written to the terminal and a directory for each test which is defined
in the configuration file. Each of them contains the log of the emerge process
(if any) and a directory for each implementation found, which in turn contains
a log for the pkg-config process, one for the compilation of the test suite,
one that checks whether the linking is correct and finally the test suit
execution log.
.PP
The logs are present only if the related process has been performed. For
instance, no log is available for the emerge process if the package was already
compiled. Please refer to the main.log file to see which processes have been
skipped.
|
GitWeb