Experiments

This page describes experiments conducted mainly with dlvhex version 2.x. It contains information how to obtain and build dlvhex 2.x, as well as measurements for

dlvhex 2.x

dlvhex 2.x is not yet released, but you can get it from sourceforge using subversion:

Test Instances and Detailed Benchmark Results:

DMCS Instances

These test instances were created using the DMCS benchmark instance generator and converted to the input format of MCS-IE using generate_master_file.py. Please refer to the original DMCS benchmark instance generator page for details of topologies and instance parameters.

Download

All DMCS instances are contained in the following archive: dmcs_instances.tar.bz2.

Complete and Detailed Results






dlvhex 1.x dlvhex 2.x




(dlv backend)

clingo backend dlv backend
topology parameters instance #models exit status memory used (MB) time used (s) #units #unit-
dependencies
memory used (MB) time used (s) memory used (MB) time used (s)
diamond 7-7-3-3 a 4 OK 11.1 1.91 17 35 5.8 0.16 5.5 0.22
b 16 ERR 3000 600 17 34 9.8 0.28 9.9 0.38
c 4 OK 70.6 4.89 16 32 7.6 0.24 6.1 0.31
d 24 OK 21.5 1.88 17 34 5.9 0.14 4.5 0.17
e 12 OK 81 4.73 17 33 7.5 0.22 7.7 0.27
f 24 OK 118.3 5.48 16 32 6.1 0.17 5.4 0.22
g 28 OK 83.9 5.3 17 35 5.9 0.19 5.9 0.22
h 8 OK 912.9 23.15 16 32 7.2 0.25 9.1 0.35
i 32 OK 566.6 22.99 17 36 7.6 0.26 8 0.32
j 28 OK 1257.3 276.8 16 32 8.3 0.3 8.1 0.41
7-7-4-4 a 36 ERR 3000 600 17 34 12.3 0.56 11.9 0.87
b 12 OK 2017.2 80.2 17 34 8 0.25 6.6 0.37
c 48 ERR 1012.6 600 17 36 21 1.23 26.5 1.71
d 14 ERR 3000 600 17 34 11.3 0.43 9.7 0.67
e 16 ERR 547.8 600 16 32 10 0.48 12.8 0.63
f 40 ERR 466 600 17 35 14.4 0.68 14.4 1.23
g 16 ERR 3000 600 17 35 10.8 0.47 13 0.56
h 768 ERR 882.8 600 16 33 34.5 1.23 32 2.21
i 122 OK 1666.6 348.14 17 36 10.2 0.39 18 0.61
j 32 OK 198.9 6.96 16 32 7.4 0.22 6 0.26
7-7-5-5 a 48 ERR 3000 600 17 34 54.9 4.01 90.1 11.08
b 16 ERR 3000 600 17 35 36.7 3.08 70.4 5.45
c 336 ERR 449.6 600 16 32 18.7 0.77 22.8 1.14
d 108 ERR 3000 600 17 34 36.9 2.13 64.2 4.89
e 32 ERR 800.2 600 17 35 15.3 0.9 15.7 1.12
f 32 ERR 752.3 600 16 32 15.9 0.8 33 1.11
g 112 ERR 2267.9 600 16 32 45.9 1.44 23.3 2.37
h 64 ERR 3000 600 17 36 24.2 1.32 66.1 2.24
i 24 ERR 457.5 600 17 35 14.9 0.75 15 1.02
j 12 ERR 413.4 600 16 31 18 0.56 11.3 0.89
house 9-9-3-3 a 812 ERR 3000 600 19 38 23.3 0.86 59.6 0.97
b 68 ERR 956.5 600 20 41 15 0.51 16 0.71
c 384 ERR 491.7 600 20 40 12.9 0.47 11.6 0.55
d 48 ERR 398.5 600 20 41 8 0.24 7 0.38
e 84 ERR 482.8 600 19 37 10.2 0.34 9.2 0.39
f 200 ERR 1588.2 600 19 38 12.7 0.46 17.6 0.68
g 384 ERR 812.9 600 20 42 13.4 0.46 15 0.61
h 384 ERR 3000 600 20 41 39.3 1.33 39.7 1.99
i 1344 ERR 3000 600 19 39 54.1 1.21 32.9 1.32
j 84 ERR 3000 600 20 42 12.8 0.67 13.5 1.07
9-9-4-4 a 512 ERR 3000 600 19 39 44.5 2.91 77.5 4.78
b 1920 ERR 3000 600 20 43 87.9 3.77 259.3 13.78
c 20 ERR 3000 600 20 42 67.7 3.01 109.5 6.89
d 40 ERR 3000 600 20 42 11.8 0.58 11.6 0.87
e 846 ERR 3000 600 20 41 24.6 0.72 23.1 0.89
f 60 ERR 3000 600 20 41 35.1 2.03 71.2 3.88
g 144 ERR 3000 600 20 41 30.3 0.99 19.8 1.5
h 368 ERR 3000 600 20 41 31.3 1.56 50.3 2.41
i 200 ERR 3000 600 20 41 87.9 3.37 250.2 10.13
j 19152 ERR 3000 600 20 42 514.9 142.21 332.4 132.02
ring 7-7-4-4 a 288 ERR 854.5 600 15 28 16.4 0.55 10.8 0.6
b 12 ERR 933.8 600 15 28 13 0.43 8.2 0.46
c 288 ERR 3000 600 15 28 16 0.62 18.4 0.99
d 456 ERR 3000 600 15 28 28.7 0.77 22.3 1.12
e 80 ERR 3000 600 15 28 12.7 0.76 15.4 0.8
f 324 ERR 3000 600 15 28 22.5 1.2 30.2 1.23
g 24 ERR 905.7 600 15 28 8.5 0.41 7.5 0.39
h 376 ERR 893.8 600 15 28 17.7 0.49 11.8 0.59
i 288 ERR 3000 600 15 28 25 1.26 42.6 2.37
j 128 ERR 494 600 15 28 9 0.38 7.1 0.41
7-7-5-5 a 24 ERR 3000 600 15 28 21.6 0.93 16.6 0.88
b 16 ERR 3000 600 15 28 16.5 0.98 27.5 1.33
c 172 ERR 3000 600 15 28 42.8 1.7 20.3 1.89
d 40 ERR 3000 600 15 28 25.2 1.61 55 2.22
e 156 ERR 3000 600 15 28 18.9 1.49 48.6 1.53
f 14 ERR 3000 600 15 28 20.7 1.43 48.1 2.27
g 102 ERR 3000 600 15 28 32.5 1.4 68.8 1.91
h 224 ERR 3000 600 15 28 67.4 2.07 47.5 2.57
i 840 ERR 3000 600 15 28 34.8 1.78 74.9 2.65
j 3 ERR 3000 600 15 28 17 0.5 10.2 0.55
7-8-5-5 a 256 ERR 3000 600 15 28 34.3 1.1 24.3 1.05
b 40 ERR 3000 600 15 28 57 1.43 24.5 1.73
c 264 ERR 3000 600 15 28 24.1 0.73 14.1 0.8
d 184 ERR 3000 600 15 28 33.9 1.33 39.2 2.09
e 24 OK 2947.3 295.19 15 28 10.7 0.38 6.9 0.32
f 864 ERR 3000 600 15 28 53.1 1.6 31.1 2.13
g 120 ERR 1910.9 600 15 28 16.2 0.72 11.8 0.98
h 1344 ERR 3000 600 15 28 38.8 1.95 62.9 3.53
i 3276 ERR 3000 600 15 28 139.7 2.95 120.8 4.36
j 168 ERR 3000 600 15 28 15.9 1.33 14.2 0.97
7-9-5-5 a 54 ERR 3000 600 15 28 45.5 3.31 39.2 4.87
b 116 ERR 3000 600 15 28 38.7 1.49 18.4 1.93
c 400 ERR 3000 600 15 28 61.6 1.4 67 1.88
d 608 ERR 3000 600 15 28 23.7 0.93 23.2 1.24
e 32 ERR 3000 600 15 28 28.2 2.1 46.2 3.92
f 40 ERR 3000 600 15 28 12.3 0.8 15.7 1.03
g 576 ERR 3000 600 15 28 25.7 1.65 74.2 2.19
h 336 ERR 3000 600 15 28 32.5 1.62 24.9 1.93
i 234 ERR 3000 600 15 28 60.5 2.73 42.5 4.12
j 272 ERR 3000 600 15 28 20.8 1.42 29.1 1.57
8-7-5-5 a 54 ERR 2219.2 600 17 32 8.7 0.33 7.2 0.5
b 138 ERR 3000 600 17 32 53.1 3.67 85.1 4.65
c 36 ERR 3000 600 17 32 11.4 0.5 10.1 0.65
d 672 ERR 3000 600 17 32 108.7 4.16 58.5 6.88
e 428 ERR 3000 600 17 32 39.1 2.29 46.6 3.39
f 1440 ERR 3000 600 17 32 50.7 1.86 37.7 2.35
g 168 ERR 3000 600 17 32 61.3 2.21 79.4 2.97
h 168 ERR 3000 600 17 32 18 1.39 25.3 2.16
i 240 ERR 3000 600 17 32 41.8 1.39 26.6 1.74
j 64 ERR 3000 600 17 32 10.7 0.56 13.1 0.75
8-8-5-5 a 760 ERR 3000 600 17 32 28.7 1.64 56 1.81
b 312 ERR 3000 600 17 32 25 1.55 25.6 2.05
c 72 ERR 3000 600 17 32 22.6 1.3 23.7 1.96
d 520 ERR 3000 600 17 32 39.1 3.02 55.1 4.72
e 364 ERR 3000 600 17 32 19.1 1.18 42.7 1.39
f 2400 ERR 3000 600 17 32 116.2 2.36 80.7 3.87
g 528 ERR 3000 600 17 32 91.2 2.96 44.1 4.09
h 1812 ERR 3000 600 17 32 75.2 2.56 67.3 2.52
i 192 ERR 3000 600 17 32 15 0.93 16 1.33
j 180 ERR 3000 600 17 32 14.4 0.65 14.3 1.12
zigzag 7-7-3-3 a 8 OK 61 11.63 16 34 9.9 0.35 14.1 0.42
b 28 ERR 3000 600 17 36 13.8 0.44 9.6 0.77
c 128 OK 258.3 6.61 17 36 9.2 0.21 6.9 0.24
d 184 ERR 3000 600 17 34 17.1 0.55 14.2 1.13
e 64 OK 1255.2 362.68 16 35 9.5 0.38 10 0.56
f 42 OK 351.8 73.76 17 35 13.1 0.4 9.2 0.64
g 6 OK 246.3 12.09 17 36 7.5 0.27 6.8 0.38
h 10 OK 465.1 74.19 17 35 9.1 0.31 10.1 0.45
i 8 OK 42.9 4.13 17 33 5.8 0.18 9.3 0.24
j 14 OK 574.9 19.7 16 33 10.9 0.34 9.8 0.38
7-7-4-4 a 80 ERR 3000 600 16 34 69 2.38 260.5 6.81
b 36 ERR 1640.2 600 17 38 15.6 0.75 14.3 1.08
c 24 ERR 855.5 600 16 33 19.4 1.07 54.8 1.99
d 56 OK 674.2 265.04 17 35 11.5 0.43 16.5 0.53
e 36 ERR 894.7 600 17 38 21.6 1.26 32.8 1.83
f 192 ERR 3000 600 16 33 52.9 1.84 89.8 3.1
g 134 ERR 377 600 16 35 23.8 0.71 16.8 0.89
h 22 ERR 251.3 600 15 31 12.1 0.55 12.8 0.84
i 18 OK 451 69.89 17 37 9 0.37 8.5 0.48
j 4 ERR 473.6 600 16 34 17.9 0.66 20.5 1.07
7-7-5-5 a 112 ERR 2813.4 600 16 33 29.5 1.59 51.1 3.1
b 32 ERR 3000 600 16 33 101.4 9.77 291.8 44.59
c 56 ERR 1763.9 600 16 35 64.8 1.83 117.1 4.36
d 64 ERR 495.3 600 16 33 18.7 1 29.4 1.76
e 80 ERR 3000 600 16 34 27.9 1.89 46.5 3.1
f 42 ERR 3000 600 17 36 45.1 2.74 75.9 6.34
g 32 ERR 3000 600 17 38 29.5 1.77 38.8 4.18
h 8 ERR 872 600 17 38 18.9 1.2 52.7 2.35
i 32 ERR 1824.4 600 16 34 43.6 2.21 55.1 5.78
j 12 ERR 3000 600 17 39 35 2.84 42.4 4.48
minimum

3 - 11.1 1.88 15 28 5.8 0.14 4.5 0.17
maximum

19152 - 3000 600 20 43 514.9 142.21 332.4 132.02
average

387.84 - 2118.48 515.55 16.63 33.09 32.1 2.29 40.18 3.24
standard deviation

1656.14 - 1152.95 196.75 1.52 4.25 47.51 11.93 52.77 11.69


Reviewer Selection 1 - Instances


These test instances were created using revsel2.cpp with varied parameter -t (the number of conference track per instance).

Download

All instances are contained in the following archive: revsel1_instances.tar.bz2.

Complete and Detailed Results




dlvhex 1.x (old heuristics H1 + dlv backend) dlvhex 2.x + former heuristics H1 + dlv backend dlvhex 2.x + simple heuristics H2 + dlv backend
# tracks # papers exit status memory used (MB) time used (s) exit status memory used (MB) time used (s) exit status memory used (MB) time used (s)
1 20 OK 43.9 1.49 OK 31.7 1.45 OK 31.5 1.36
2 20 OK 73.1 3.31 OK 72.7 2.65 OK 56.2 2.11
3 20 OK 120.8 5.89 OK 112.1 4.17 OK 82 3.15
4 20 OK 152.8 11.64 OK 142.4 5.82 OK 106.3 4.02
5 20 OK 197.5 28.47 OK 180.2 8.21 OK 131.3 5.25
6 20 OK 239.7 71.64 OK 239.6 11.92 OK 165.2 6.17
7 20 OK 325.8 109.14 OK 258.5 16.31 OK 164.5 7.17
8 20 OK 777.3 477.51 OK 309.4 28.08 OK 213.7 7.92
9 20 ERR 1716.3 600 OK 422.4 51.74 OK 250.1 9.58
10 20 ERR 2533.7 600 OK 588.3 100.26 OK 242.2 10.87
11 20 ERR 3000 600 OK 1413.7 206.24 OK 262.4 12.01
12 20 ERR 3000 600 OK 3000 458.02 OK 294.3 12.65
13 20 ERR 3000 600 OK 3000 600 OK 337.9 13.95
14 20 ERR 3000 600 ERR 3000 600 OK 295.7 15.03
15 20 ERR 3000 600 ERR 3000 600 OK 397.8 16.73
16 20 ERR 3000 600 ERR 3000 600 OK 380.5 17.32
17 20 ERR 3000 600 ERR 3000 600 OK 461.4 18.85
18 20 ERR 3000 600 ERR 3000 600 OK 481.2 19.9
19 20 ERR 3000 600 ERR 3000 600 OK 520.3 21
20 20 ERR 3000 600 ERR 3000 600 OK 444.4 21.94
21 20 ERR 3000 600 ERR 3000 600 OK 561.5 23.11
22 20 ERR 3000 600 ERR 3000 600 OK 605.1 24.48
23 20 ERR 3000 600 ERR 3000 600 OK 628.5 25.54
24 20 ERR 3000 600 ERR 3000 600 OK 643.3 27.07
25 20 ERR 3000 600 ERR 3000 600 OK 683.8 28.29
26 20 ERR 3000 600 ERR 3000 600 OK 694.6 29.74
27 20 ERR 3000 600 ERR 3000 600 OK 726 31.09
28 20 ERR 3000 600 ERR 3000 600 OK 756.9 31.94
29 20 ERR 3000 600 ERR 3000 600 OK 745 33.45
30 20 ERR 3000 600 ERR 3000 600 OK 761.2 34.26
31 20 ERR 3000 600 ERR 3000 600 OK 848.4 36.66
32 20 ERR 3000 600 ERR 3000 600 OK 894.2 37.29
33 20 ERR 3000 600 ERR 3000 600 OK 841.8 39.43
34 20 ERR 3000 600 ERR 3000 600 OK 914.3 39.57
35 20 ERR 3000 600 ERR 3000 600 OK 889.9 42.45
36 20 ERR 3000 600 ERR 3000 600 OK 946.1 43.3
37 20 ERR 3000 600 ERR 3000 600 OK 1019 44.7
38 20 ERR 3000 600 ERR 3000 600 OK 998.6 45.95
39 20 ERR 3000 600 ERR 3000 600 OK 1015 46.23
40 20 ERR 3000 600 ERR 3000 600 OK 1118.2 47.89
41 20 ERR 3000 600 ERR 3000 600 OK 1051.6 49.73
42 20 ERR 3000 600 ERR 3000 600 OK 1113.6 52.66
43 20 ERR 3000 600 ERR 3000 600 OK 1213.4 53.47
44 20 ERR 3000 600 ERR 3000 600 OK 1291 54.54
45 20 ERR 3000 600 ERR 3000 600 OK 1358.2 56.84
46 20 ERR 3000 600 ERR 3000 600 OK 1287.4 58.74
47 20 ERR 3000 600 ERR 3000 600 OK 1347.6 60.08
48 20 ERR 3000 600 ERR 3000 600 OK 1286.4 61.41
49 20 ERR 3000 600 ERR 3000 600 OK 1403.1 62.61
50 20 ERR 3000 600 ERR 3000 600 OK 1408.2 65


Reviewer Selection 2 - Instances


These test instances were created using revsel1.cpp with varied parameter -p (the number of papers per conference track).

Download

All instances are contained in the following archive: revsel2_instances.tar.bz2.

Complete and Detailed Results





dlv clingo (clasp+gringo) dlvhex 2.x + simple heuristics H2 + dlv backend dlvhex 2.x + simple heuristics H2 + clingo backend
#tracks #papers #models exit
status
memory
used
(MB)
time
used
(s)
exit
status
memory
used
(MB)
time
used
(s)
exit status memory
used
(MB)
time
used
(s)
exit
status
memory
used
(MB)
time
used
(s)
5 1 0 OK 0 0 OK 0 0 OK 0 0.01 OK 0 0.01
5 2 9 OK 0 0 OK 0 0 OK 0 0.04 OK 0 0.03
5 3 9 OK 0 0 OK 0 0.01 OK 0 0.05 OK 0 0.03
5 4 9 OK 0 0.01 OK 0 0.01 OK 0 0.06 OK 0 0.06
5 5 9 OK 0 0.03 OK 0 0.02 OK 0 0.07 OK 0 0.06
5 6 9 OK 0 0.06 OK 0 0.04 OK 4.2 0.13 OK 4.5 0.12
5 7 9 OK 0 0.11 OK 0 0.04 OK 5.1 0.16 OK 4.7 0.13
5 8 9 OK 4.4 0.18 OK 5.4 0.1 OK 6.1 0.22 OK 4.9 0.17
5 9 9 OK 7.2 0.29 OK 6.4 0.19 OK 8.1 0.33 OK 5 0.26
5 10 9 OK 9.8 0.45 OK 6.4 0.2 OK 8.8 0.47 OK 5.3 0.28
5 11 9 OK 14.6 0.66 OK 9.6 0.3 OK 14.9 0.73 OK 5.8 0.35
5 12 9 OK 27.5 1.02 OK 14.3 0.42 OK 22.7 0.88 OK 6.5 0.63
5 13 9 OK 33.5 1.29 OK 18.3 0.61 OK 26.6 1.1 OK 7.4 0.63
5 14 9 OK 42.8 1.82 OK 22.6 0.81 OK 29.2 1.45 OK 8.1 0.67
5 15 9 OK 57.8 2.28 OK 28.8 1.1 OK 38.8 1.78 OK 9.3 1.79
5 16 9 OK 69.3 2.97 OK 36.4 1.47 OK 51.2 2.44 OK 10.8 1.62
5 17 9 OK 116.8 3.88 OK 44.2 1.91 OK 58.6 2.87 OK 12.6 1.54
5 18 9 OK 125.1 5.14 OK 54.5 2.49 OK 79.7 3.47 OK 14.2 1.77
5 19 9 OK 151.1 6.93 OK 66.2 3.21 OK 97.3 4.16 OK 16.7 2.39
5 20 9 OK 230.8 8.57 OK 80.6 3.92 OK 112.8 5.48 OK 18.7 2.99
5 21 9 OK 231.7 10.18 OK 95.1 5.04 OK 155.6 6.41 OK 22.2 3.76
5 22 9 OK 256.3 10.59 OK 114.4 6.02 OK 180.1 7.63 OK 25.1 4.73
5 23 9 OK 330.8 13.45 OK 133.6 7.43 OK 201.9 9.99 OK 30.2 6.99
5 24 9 OK 458.6 15.71 OK 162.1 8.96 OK 220 10.9 OK 33.2 6.43
5 25 10 OK 459.8 17.77 OK 186.5 10.52 OK 243.1 12.8 OK 38.8 10.09
5 26 11 OK 520.6 21.97 OK 214.2 13.1 OK 341.5 14.71 OK 44.5 8.89
5 27 12 OK 600.1 25.41 OK 251.1 15.06 OK 385.5 17.68 OK 49.1 12.53
5 28 13 OK 911.8 28.1 OK 290.7 17.63 OK 527.2 19.97 OK 55.9 13.59
5 29 14 OK 913.3 35.46 OK 331.7 20.5 OK 432.7 23.43 OK 64.5 13.22
5 30 15 OK 914.9 36.62 OK 380.8 23.93 OK 398 26.79 OK 70.6 14.61
5 31 16 OK 1013.5 44.53 OK 427.4 27.78 OK 519.4 28.94 OK 80.1 20.39
5 32 17 OK 1227.8 48.05 OK 492.9 32.12 OK 295.6 33.01 OK 92.7 23.68
5 33 18 OK 1816.3 60.42 OK 549.8 37.41 OK 947.6 39.03 OK 101.4 24.63
5 34 19 OK 1818.4 63.68 OK 623.1 44.07 OK 821.5 43.31 OK 117.1 32.27
5 35 20 OK 1820.5 72.94 OK 691.1 48.91 OK 822 49.35 OK 132.3 31.37
5 36 21 OK 1958.9 78.54 OK 779.3 55.41 OK 683.3 52.84 OK 141.8 36.52
5 37 22 OK 2107.6 92.86 OK 865.3 63.04 OK 685.3 61.2 OK 157.6 39.54
5 38 23 OK 2329.9 99.25 OK 951.3 75.02 OK 939.4 68.49 OK 172.6 45.87
5 39 24 OK 2794.8 117.66 OK 1045.2 80.86 OK 1211.5 76.46 OK 194.9 49.82
5 40 25 ERR 3000 600 OK 1134.8 90.33 OK 855.4 83.5 OK 210.1 65.25
5 41 26 ERR 3000 600 OK 1296.2 105.07 OK 488.3 93.92 OK 233.4 67.82
5 42 27 ERR 3000 600 OK 1383.3 113.2 OK 788.1 97.51 OK 251 73.18
5 43 28 ERR 3000 600 OK 1520.5 127.61 OK 1234.9 116.37 OK 280.6 81.97
5 44 29 ERR 3000 600 OK 1644.1 143.91 OK 1723.9 121.46 OK 308.9 89.94
5 45 30 ERR 3000 600 OK 1783.1 159.53 OK 1372.8 133.25 OK 328.6 100.56
5 46 31 ERR 3000 600 OK 1952.1 172.73 OK 1809.8 149.21 OK 354.1 118.54
5 47 32 ERR 3000 600 OK 2094.7 202.25 OK 2091.4 167.52 OK 386.1 121.87
5 48 33 ERR 3000 600 OK 2261.4 215.39 OK 1431.8 175.74 OK 423.5 142.17
5 49 34 ERR 3000 600 OK 2503.8 242.77 OK 1991.7 194.42 OK 460.9 155.78
5 50 35 ERR 3000 600 OK 2890.7 262.91 OK 2061.6 206.17 OK 497.9 193.36
5 51 36 ERR 3000 600 OK 2943.4 300.58 OK 1598.3 225.69 OK 531.6 203.94
5 52 37 ERR 3000 600 ERR 3000 600 OK 2211.3 236.35 OK 583.2 207.76
5 53 38 ERR 3000 600 ERR 3000 600 OK 2340.5 261.56 OK 641.7 214.53
5 54 39 ERR 3000 600 ERR 3000 600 OK 2384.7 278.85 OK 684.4 265.44
5 55 40 ERR 3000 600 ERR 3000 600 OK 2313.8 309.97 OK 720.5 280.53
5 56 41 ERR 3000 600 ERR 3000 600 OK 2598.7 324.43 OK 764.2 288.02
5 57 42 ERR 3000 600 ERR 3000 600 OK 2447.8 350.54 OK 814.7 370.84
5 58 43 ERR 3000 600 ERR 3000 600 OK 1972.2 392.71 OK 868.2 340.77
5 59 44 ERR 3000 600 ERR 3000 600 OK 2197.3 425.45 OK 951.9 425.85
5 60 45 ERR 3000 600 ERR 3000 600 OK 2074.8 437.47 OK 1002.4 411.15


External Behavior Learning


This section compares dlvhex 2.x with and without external behavior learning (EBL).

General Information

All benchmarks were carried out on a machine with 24 Dual-Core AMD Opteron Magny-Cours 6176 SE CPUs and 128 GB RAM, running Linux and using Clasp 2.0.5 as solver backend.

Set Partitioning

In our first (synthetic) example program we consider a set S of elements which is partitioned into two subsets S1 and S2, such that S1 contains at most two elements. This is realized with HEX-programs using external atoms of kind &setminus[p,q](X), which computes the set difference of the extensions of p and q.

We use the program template partitioning.hex, specifying two cyclic rules which realize the partitioning, and give the solver a hint on monotonicity in the first input predicate. For benchmarking we use a benchmark script which automatically iterates through instance sizes and instantiates the program template accordingly. The script can be downloaded here (and is also contained in the dlvhex repository): benchmark.sh. It is called like this:

benchmark.sh [Program] [Min Domain Size] [Max Domain Size] [Timeout/s] [Configuration Strings (optional)],

where

The benchmark script evaluates Program for each integer i between Min Domain Size and Max Domain Size, and adds all facts domain(1), ..., domain(i) to the program prior to evaluation. dlvhex is then called once per configuration specified in the last argument.

An example call is:

./benchmark.sh partitioning.hex 1 20 300 "--solver=genuinegc;--solver=genuinegc --extlearn"

It evaluates partitioning.hex for each instance size from 1 to 20, once with arguments --solver=genuinegc and once with arguments --solver=genuinegc --extlearn, i.e., it compares the genuine solver (with gringo+clasp backend) with and without external learning.

We summarized the runtimes for the configurations

in benchmarks.ods.

Bird-Penguin

We use DL-programs for realizing default reasoning over a description logic knowledge base, realized as a plugin to dlvhex. Program birdpenguin.hex queries all birds from ontology (template) bird.owl and encodes that bird typically fly (if the contrary cannot be derived). For varying the input size, we use an adopted version of the benchmark script above which produces ontology instances with A-boxes of different sizes. The script can be found in the repository of the DL-plugin

The runtime results of 5 runs are summarized in birdpenguin.ods.

Wine Ontology

The findings from the bird-penguin example carry over to the larger wine ontology http://www.w3.org/TR/owl-guide/wine.rdf (and scaled variants available at http://kaon2.semanticweb.org/download/test_ontologies.zip). We made three experiments which implement three different default rules as follows:

1. "A wine is dry, unless the contrary can be derived"

For this experiment we used three scripts drywine_pi.sh, drywine_omega.sh and drywine_y.sh to produce HEX programs for classifying different categories of wines, based on different transformations [dek2009].

2. "A wine is dry, unless it is known that it is sweet"

For this experiment we used the Omega-transformation, which turned out to be the most efficient one. We used the script sweetwine.sh to produce the HEX programs for classifying different categories of wines.

3. "A wine is white, unless it is known to be red"

For this experiment we used the Omega-transformation, which turned out to be the most efficient one. We used the script redwine.sh to produce the HEX programs for classifying different categories of wines.

The runtime results of 3 runs are summarized in benchmarks.ods.

Multi-context Systems

For our multi-context benchmarks we used the instances from mcs.zip and the MCS plugin for dlvhex.

Runtime results are summarized in benchmarks.ods.

Literature

Minh Dao-Tran, Thomas Eiter, and Thomas Krennwallner.
Realizing Default Logic over Description Logic Knowledge Bases.
In Claudio Sossai and Gaetano Chemello, editors, Proceedings of the 10th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU 2009), Verona, Italy, July 1-3, 2009 , volume 5590 of LNAI, pages 602-613. Springer, July 2009.
DOI ]

 

General
dlvhex source code @ github.com
Description-Of-A-Project

Popular Plugins
Action Plugin
DecisionDiagrams Plugin
Description Logics Plugin
Description Logics Lite Plugin
MELD: Belief Merging Plugin
Nested HEX Plugin
MCSIE Plugin
String Plugin
dlvhex-semweb Project

Documentation
User Guide
README
doxygen
Writing Plugins in C++
Writing Plugins in Python