[cc65] Benchmarks

For the new release, I would like to have a few benchmarks that generate
reproducable numbers that may be used to compare the new cc65 version to the
old one - or to other 6502 C compilers.

I'm currently uncertain if these should go with the compiler or be published
separately. Here are my ideas about how a usable benchmark could look:

  1. Should not be written for cc65.

  2. Should not be too platform specific, because that does usually imply non
     portable C code. Use of programs using conio.h or similar is limited,
     because its cc65 specific.

  3. License should allow source code changes and unlimited distribution.

What I have is

  a. the sieve benchmark sample, that comes with cc65. It violates 1. and
     uses somewhat non-portable code for time measurement which can be
     fixed.

     The C64 numbers for this benchmark are (smaller is better):

       version       size  code generation options                  results
      ------------------------------------------------------------------------
       cc65 2.12.0   3799  -Oirs --codesize 500                     2.116 sec
       cc65 head     3883  -Oirs --codesize 500                     2.000 sec

  b. The dhrystone benchmark. I'm not clear about 3. and it would need a
     makefile and some small changes for better time measurement, but
     apart from that, it's ok.

     The C64 numbers for this benchmark are (larger is better):

       version       size  code generation options                  results
      ------------------------------------------------------------------------
       cc65 2.12.0   6401  -Oirs                                    48 d/sec
       cc65 2.12.0   6531  -Oirs -DREG=register                     49 d/sec
       cc65 2.12.0   6661  -Oirs --codesize 400 -DREG=register      52 d/sec

       cc65 head     6645  -Oirs                                    73 d/sec
       cc65 head     6690  -Oirs -DREG=register                     74 d/sec
       cc65 head     6684  -Oirs --codesize 400 -DREG=register      75 d/sec

     Please note how the executable gets *smaller* even with --codesize 400.
     Additional inlining will also allow the optimizer to see more of the
     code.

  c. Someone posted a link to a program calculating pi up to 1000 digits. Same
     problems as with b. Output of the result on the console is also part of
     the measured code, which should probably be changed, because measuring 
     the speed at which text is printed on the console is somewhat boring. 

     The program needs --static-locals, because it allocates large arrays on
     the stack. This does also speed up the code somewhat, so to be
     comparable, allocation of these variables should be changed.

     The C64 numbers for this benchmark are (smaller is better):

       version       size  code generation options                  results
      ------------------------------------------------------------------------
       cc65 2.12.0   6287  -Oirs --static-locals                    35:27 min
       cc65 head     6140  -Oirs --static-locals                    29:22 min


If you have any program that would qualify as a benchmark program and complies
with 1-3, please let me know.

Regards


        Uz


-- 
Ullrich von Bassewitz                                  uz@musoftware.de
----------------------------------------------------------------------
To unsubscribe from the list send mail to majordomo@musoftware.de with
the string "unsubscribe cc65" in the body(!) of the mail.