LeakWatch

Command Line Output

With sufficient data to produce an estimate

LeakWatch's output for a class whose main method likely contains an information leak may look like this:

Stopped after 220 executions: corrected leakage: 0.1231 bits
There IS evidence of an information leak (estimated range: 0.0525 - 0.1938 bits).
  • The first line states the estimated leakage from the program's secret information (identified with LeakWatchAPI.secret()) to its publicly-observable information (identified with LeakWatchAPI.observe()) — "corrected" indicates that LeakWatch has attempted to account for bias in the estimate.
  • The second line confirms whether or not a leak exists, given the information that LeakWatch has collected. If LeakWatch estimates that a leak does exist, the size of the leak will be bounded (if possible).

Alternatively, LeakWatch's output for a class whose main method does not contain an information leak may look like this:

Stopped after 216 executions: corrected leakage: 0.0000 bits
There is no evidence of an information leak.

With insufficient data to produce an estimate

Under some circumstances — for example, if a specific number of executions are performed using the -n K (or --executions=K) option and this number is not large enough — LeakWatch may be unable to produce an estimate of the amount of information that leaks in the class's main method. In these situations, LeakWatch's output explains what you should do; e.g.:

Terminated: insufficient data collected to estimate leakage:
* not enough executions were completed; try increasing the value of -n/--executions

With the -i (or --interval) option

If the -i K (or --interval=K) option is specified, LeakWatch will produce an additional line of output for every K executions of the class's main method; each line shows LeakWatch's intermediate estimates after that number of executions of the main method. For example, if -i 50 is specified, LeakWatch's output may look like this:

After 50 executions: observed leakage 0.1409 bits, corrected leakage 0.0976 bits; upper bound for zero leakage: 0.1627 bits
After 100 executions: observed leakage 0.1258 bits, corrected leakage 0.1042 (+/- 0.10386) bits; upper bound for zero leakage: 0.0813 bits
After 150 executions: observed leakage 0.1190 bits, corrected leakage 0.1046 (+/- 0.08439) bits; upper bound for zero leakage: 0.0542 bits
After 200 executions: observed leakage 0.1304 bits, corrected leakage 0.1196 (+/- 0.07557) bits; upper bound for zero leakage: 0.0407 bits

If there is not enough data available to make a meaningful estimate after a particular number of executions, LeakWatch will output a suitable message: "not enough executions", "not enough unique secret values", and/or "not enough unique observable values".

If enough data is available, LeakWatch will output:

  • an initial estimate of the leakage from the secret to the publicly-observable information, based on the data collected by LeakWatch ("observed leakage");
  • if mutual information is being estimated:
    • a corrected estimate of the observed leakage, after taking into account the bias that may be present in the data ("corrected leakage");
    • bounds on the corrected estimate, if there is enough data available to calculate a 95% confidence interval;
    • a value below which the observed leakage should be considered consistent with no leakage ("upper bound for zero leakage").