14.6. Statistic Per Single Benchmark (Graph And CSV)

14.6.1. Enable A Single Statistic

A single statistic is a statics for 1 dataset for 1 solver configuration. Unlike a problem statistic, it does not aggregate over solver configurations.
The benchmarker supports outputting single statistics as graphs and CSV (comma separated values) files to the benchmarkDirectory. To configure one, add a singleStatisticType line: 
<plannerBenchmark>
  <benchmarkDirectory>local/data/nqueens/solved</benchmarkDirectory>
  <inheritedSolverBenchmark>
    <problemBenchmarks>
      ...
      <problemStatisticType>...</problemStatisticType>
      <singleStatisticType>PICKED_MOVE_TYPE_BEST_SCORE_DIFF</singleStatisticType>
    </problemBenchmarks>
    ...
  </inheritedSolverBenchmark>
  ...
</plannerBenchmark>
Multiple singleStatisticType elements are allowed.
Note
These statistic per single benchmark can slow down the solver noticeably, which affects the benchmark results. That's why they are optional and not enabled by default.
The following types are supported:

14.6.2. Constraint Match Total Best Score Over Time Statistic (Graph And CSV)

To see which constraints are matched in the best score (and how much) over time, add: 
    <problemBenchmarks>
      ...
      <singleStatisticType>CONSTRAINT_MATCH_TOTAL_BEST_SCORE</singleStatisticType>
    </problemBenchmarks>

Figure 14.8. Constraint Match Total Best Score Diff Over Time Statistic

Constraint Match Total Best Score Diff Over Time Statistic
Requires the score calculation to support constraint matches. Drools score calculation supports constraint matches automatically, but incremental Java score calculation requires requires more work.
Warning
The constraint match total statistics has been seen to affect the solver noticeably.

14.6.3. Constraint Match Total Step Score Over Time Statistic (Graph And CSV)

To see which constraints are matched in the step score (and how much) over time, add: 
    <problemBenchmarks>
      ...
      <singleStatisticType>CONSTRAINT_MATCH_TOTAL_STEP_SCORE</singleStatisticType>
    </problemBenchmarks>

Figure 14.9. Constraint Match Total Step Score Diff Over Time Statistic

Constraint Match Total Step Score Diff Over Time Statistic
Requires the score calculation to support constraint matches. Drools score calculation supports constraint matches automatically, but incremental Java score calculation requires requires more work.
Warning
The constraint match total statistics has been seen to affect the solver noticeably.

14.6.4. Picked Move Type Best Score Diff Over Time Statistic (Graph And CSV)

To see which move types improve the best score (and how much) over time, add: 
    <problemBenchmarks>
      ...
      <singleStatisticType>PICKED_MOVE_TYPE_BEST_SCORE_DIFF</singleStatisticType>
    </problemBenchmarks>

Figure 14.10. Picked Move Type Best Score Diff Over Time Statistic

Picked Move Type Best Score Diff Over Time Statistic

14.6.5. Picked Move Type Step Score Diff Over Time Statistic (Graph And CSV)

To see how much each winning step affects the step score over time, add: 
    <problemBenchmarks>
      ...
      <singleStatisticType>PICKED_MOVE_TYPE_STEP_SCORE_DIFF</singleStatisticType>
    </problemBenchmarks>

Figure 14.11. Picked Move Type Step Score Diff Over Time Statistic

Picked Move Type Step Score Diff Over Time Statistic