util.benchmark

Class Benchmark

java.lang.Object

util.benchmark.Benchmark

public class Benchmark
extends java.lang.Object

Benchmark is a small utility to quickly and easily run simple Benchmarks on Java code. The code is based entirely on the benchmark utility from Go (see http://golang.org/pkg/testing/).

In order to write benchmarks, you need to provide a class that has methods with signatures of the form:

 public void methodNameIrrelevant (Benchmark b) {...} 

The benchmarks in the class you wrote will be executed when the following command is run:

 java -jar Benchmark
  {name.of.class.containing.my.BenchmarkMethods} 

( The above seems to have some classloader issues ... )

Alternatively, you can run benchmarks within Java code by calling:

 Benchmark.runBenchmark(MyBenchMark.class); 

The Benchmark utility will vary the number of times to run the benchmark until it runs long enough to be measured accurately. The number of times that a benchmark is supposed to run is signaled to your benchmark function by the field N in the Benchmark argument to the function. N.B. it is the responsibility of the benchmark function you provide to run the code N number of times and not up to the Benchmark utility! A typical benchmark function will look like this:

      public static void string (Benchmark b) {
        for (int n =0; n!= b.N; ++n) {
          String sum = "";
          for (int i = 1; i!= 1000; ++i) {
            sum += i;
          }
          if (n == 0) {
            b.setBytes(sum.getBytes().length);
          }
        }
      }

  

If setBytes(...) is called in the benchmark test, the benchmarks will display information about the throughput (MB/s) of the code as well as the the average time per operation. The value passed to setBytes(...) is the number of bytes processed in a single iteration, not how many bytes were processed in b.N loop iterations.

In case the benchmark test requires lengthy setup that should be ignored, you can stop and restart the internal timer like this:

    public void myBenchmark (Benchmark b) {
      b.stopTimer();
      reallyLengthySetup();
      b.startTimer();
      for (int i =0; i!= b.N, ++i) {
        ... whatever ...
      }
    }
  

This class contains three sample benchmarks measuring the performance of string concatenation of String, StringBuffer and StringBuilder. These can be run as follows:

     $ java -jar lib/benchmark.jar benchmark.Benchmark
            string           500           6523698 ns/op           0.44 MB/s
      stringBuffer         50000             38646 ns/op          74.76 MB/s
     stringBuilder         50000             32237 ns/op          89.62 MB/s
  

The results printed are:

• name of the benchmark method
• number of times the utility ran the benchmark
• number of ns per loop
• number of MB processed per second

The tool may also be call as follows:

     $ java -jar lib/benchmark.jar benchmark.Benchmark stringB.*
      stringBuffer         50000             38646 ns/op          74.76 MB/s
     stringBuilder         50000             32237 ns/op          89.62 MB/s
  

The final argument is an optional regular expression, if provided, only benchmark methods matching this expression are run.

Field Summary

Fields

Modifier and Type	Field and Description
int	N the number of times the Benchmark utility expects you to perform whatever it is you're benchmarking.

Constructor Summary

Constructors

Constructor and Description
Benchmark()

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)
void	resetTimer() stops the timer and sets the elapsed benchmark time to zero.
util.benchmark.BenchmarkResult	run() run times the benchmark function.
static void	runBenchmark(java.lang.Class c)
static void	runBenchmark(java.lang.Class c, java.lang.String regexp)
void	setBytes(long n) records the number of bytes processed in a single operation.
void	startTimer() starts timing a test.
void	stopTimer() stops timing a test.
static void	string(Benchmark b) Example:
void	stringBuffer(Benchmark b) Example:
static void	stringBuilder(Benchmark b) Example:

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

N

public int N

the number of times the Benchmark utility expects you to perform whatever it is you're benchmarking.

A typical benchmark method will look like this:

   public void benchmarkMethod (Benchmark b) {
      for (int i = 0; i != b.N; ++i) {
        ... my benchmark code ...
      }
   }
 

The responsibility of running the benchmark N is up to you, the the utility just tells you the value of N...

Constructor Detail

Benchmark

public Benchmark()

Method Detail

runBenchmark

public static void runBenchmark(java.lang.Class c,
                                java.lang.String regexp)
                         throws java.lang.InstantiationException,
                                java.lang.IllegalAccessException

Throws:

java.lang.InstantiationException

java.lang.IllegalAccessException

runBenchmark

public static void runBenchmark(java.lang.Class c)
                         throws java.lang.InstantiationException,
                                java.lang.IllegalAccessException

Throws:

java.lang.InstantiationException

java.lang.IllegalAccessException

stringBuilder

public static void stringBuilder(Benchmark b)

Example:

   public static void stringBuilder (Benchmark b) {
     for (int n =0; n!= b.N; ++n) {
       StringBuilder sum = new StringBuilder();
       for (int i = 1; i!= 1000; ++i) {
         sum.append(i);
       }
       if (n == 0) {
         b.setBytes(sum.length());
       }
     }
   }
 

string

public static void string(Benchmark b)

Example:

   public static void string (Benchmark b) {
     for (int n =0; n!= b.N; ++n) {
       String sum = "";
       for (int i = 1; i!= 1000; ++i) {
         sum += i;
       }
       if (n == 0) {
         b.setBytes(sum.getBytes().length);
       }
     }
   }
 

main

public static void main(java.lang.String[] args)
                 throws java.lang.Throwable

Throws:

java.lang.Throwable

startTimer

public void startTimer()

starts timing a test. This function is called automatically before a benchmark starts, but it can also used to resume timing after a call to StopTimer.

stopTimer

public void stopTimer()

stops timing a test. This can be used to pause the timer while performing complex initialization that you don't want to measure.

resetTimer

public void resetTimer()

stops the timer and sets the elapsed benchmark time to zero.

setBytes

public void setBytes(long n)

records the number of bytes processed in a single operation. If this is called, the benchmark will report ns/op and MB/s.

run

public util.benchmark.BenchmarkResult run()

run times the benchmark function. It gradually increases the number of benchmark iterations until the benchmark runs for a second in order to get a reasonable measurement. It prints timing information in this form:

                testing.BenchmarkHello  100000          19 ns/op
 

stringBuffer

public void stringBuffer(Benchmark b)

Example:

   public void stringBuffer (Benchmark b) {
     for (int n =0; n!= b.N; ++n) {
       StringBuffer sum = new StringBuffer();
       for (int i = 1; i!= 1000; ++i) {
         sum.append(i);
       }
       if (n == 0) {
         b.setBytes(sum.length());
       }
     }
   }