.so init
.SH
A. MEASUREMENTS
.SH
A.1. \*(OQThe bottom line\*(CQ
.PP
Although examples often are most illustrative, the cruel world out there is
usually more interested in everyday performance figures. To satisfy those
people too, we will present a series of measurements on our code expander
taken from (close to) real life situations. These include measurements
of compile and run times of different programs,
compiled with different compilers.
.SH
A.2. Compile time measurements
.PP
Figure A.2.1 shows compile-time measurements for typical C code:
the dhrystone benchmark\(dg
.[ [
dhrystone
.]].
.FS
\(dg To be certain that we only tested the compiler and not the quality of
the code in the library, we have added our own version of
\fIstrcmp\fR and \fIstrcpy\fR and have not used the ones present in the
library.
.FE
The numbers represent the duration of each separate pass of the compiler.
The numbers at the end of each bar represent the total duration of the
compilation process. As with all measurements in this chapter, the
quoted time or duration is the sum of user and system time in seconds.
.PS
copy "pics/compile_bars"
.PE
.DS
.IP cem: 6
C to EM frontend
.IP opt:
EM peep-hole optimizer
.IP be:
EM to assembler backend
.IP cpp:
Sun's C preprocessor
.IP ccom:
Sun's C compiler
.IP iropt:
Sun's optimizer
.IP cg:
Sun's code generator
.IP as:
Sun's assembler
.IP ld:
Sun's linker
.ce 1
\fIFigure A.2.1: compile-time measurements.\fR
.DE
.sp
.PP
A close examination of the first two bars in fig A.2.1 shows that the maximum
achievable compile-time
gain compared to \fIcc\fR is about 50% for medium-sized
programs.\(dd
.FS
\(dd (cpp+ccom+as+ld)/(cem+as+ld) = 1.53
.FE
For small programs the gain will be less, due to the almost constant
start-up time of each pass in the compilation process. Only a
built-in assembler may increase this number up to
180% in the ideal case that the optimizer, backend and assembler
would run in zero time. Speed-ups of 5 to 10 times as mentioned in
.[ [
fast portable compilers
.]]
are therefore not possible on the Sun-4 family. This is also due to
Sun's implementation of saving and restoring register windows. With
the current implementation in which only a single window is saved
or restored on a register-window overflow, it is very time consuming
when programs have highly dynamic stack use
due to procedure calls (as is often the case with compilers).
.PP
Although we are currently a little slower than \fIcc\fR, it is hard to
blame this on our backend. Optimizing the backend so that it would run
twice as fast would only reduce the total compilation process by
a mere 14%.
.PP
Finally it is nice to see that our push/pop-optimization,
initially designed to generate faster code, has also increased the
compilation speed. (see also figures A.4.1 and A.4.2.)
.SH
A.3. Run time performance
.PP
Figure A.3.1 shows the run-time performance of different compilers.
All results are normalized, where the best available compiler (Sun's
compiler with full optimization) is represented by 1.0 on our scale.
.PS
copy "pics/run-time_bars"
.PE
.ce 1
\fIFigure A.3.1: run time performance.\fR
.sp 1
.PP
The fact that our compiler behaves rather poorly compared to Sun's
compiler is due to the fact that the dhrystone benchmark uses
relatively many subroutine calls; all of which have to be 'emulated'
by our backend.
.SH
A.4. Overall performance
.LP
In the next two figures we will show the combined run and compile time
performance of 'our' compiler (the ACK C frontend and our backend)
compared to Sun's C compiler. Figure A.4.1 shows the results from
measurements on the dhrystone benchmark.
.G1
frame invis left solid bot solid
label left "run time" "(in \(*msec/dhrystone)"
label bot "compile time (in sec)"
coord x 0,21 y 0,610
ticks left out from 0 to 600 by 200
ticks bot out from 0 to 20 by 5
"\(bu" at 3.5, 1000000/1700
"ack w/o opt" ljust at 3.5 + 1, 1000000/1700
"\(bu" at 2.8, 1000000/8770
"ack with opt" below at 2.8 + 0.1, 1000000/8770
"\(bu" at 16.0, 1000000/10434
"ack -O4" above at 16.0, 1000000/10434
"\(bu" at 2.3, 1000000/7270
"\fIcc\fR" above at 2.3, 1000000/7270
"\(bu" at 9.0, 1000000/12500
"\fIcc -O4\fR" above at 9.0, 1000000/12500
"\(bu" at 5.9, 1000000/15250
"\fIcc -O\fR" below at 5.9, 1000000/15250
.G2
.ce 1
\fIFigure A.4.1: overall performance on dhrystones.
.sp 1
.LP
Fortunately for us, dhrystones are not all there is. The following
figure shows the same measurements as the previous one, except
this time we took a benchmark that uses no subroutines: an implementation
of Eratosthenes' sieve:
.G1
frame invis left solid bot solid
label left "run time" "for one run" "(in sec)" left .6
label bot "compile time (in sec)"
coord x 0,11 y 0,21
ticks bot out from 0 to 10 by 5
ticks left out from 0 to 20 by 5
"\(bu" at 2.5, 17.28
"ack w/o opt" above at 2.5, 17.28
"\(bu" at 1.6, 2.93
"ack with opt" above at 1.6, 2.93
"\(bu" at 9.4, 2.26
"ack -O4" above at 9.4, 2.26
"\(bu" at 1.5, 7.43
"\fIcc\fR" above at 1.5, 7.43
"\(bu" at 2.7, 2.02
"\fIcc -O4\fR" ljust at 1.9, 1.2
"\(bu" at 2.6, 2.10
"\fIcc -O\fR" ljust at 3.1,2.5
.G2
.ce 1
\fIFigure A.4.2: overall performance on Eratosthenes' sieve.
.sp 1
.PP
Although the above figures speak for themselves, a small comment
may be in place. At first it is clear that our compiler is neither
faster than \fIcc\fR, nor produces faster code than \fIcc -O4\fR. It should
also be noted however, that we do produce better code than \fIcc\fR
at only a very small additional cost.
It is also worth noticing that push-pop optimization
increases run-time speed as well as compile speed.
The first seems rather obvious,
since optimized code is
faster code, but the increase in compile speed may come as a surprise.
The main reason is that the \fIas\fR+\fIld\fR time depends largely on the
amount of generated code, which in general
depends on the efficiency of the code.
Push-pop optimization removes a lot of useless instructions which
would otherwise
have found their way through to the assembler and the loader.
Useless instructions inserted in an early stage in the compilation
process will slow down every following stage, so elimination of useless
instructions in an early stage, even when it requires a little computational
overhead, can often be beneficial to the overall compilation speed.
.bp