Discrepancy to MadEvent w.r.t. jet rates

[kilian]

This is the message from Marcus Rammes:

Wenn ich DeltaR (also sqrt(Delta eta2 + Delta phi2)) von den b-Quarks gegen die Photonen plotte, zeigt WHIZARD ein merkwuerdiges Verhalten fuer kollineare Photonen (diff_b_gamma_angle.eps), das MadEvent? nicht aufweist. Haben Sie eine Erklaerung woran das liegen koennte?

Ich habe einen Photon-Cut von E<2 GeV und p_T<2 GeV eingestellt. Saemtliche Cuts und die PDF stimmen bei MG und WHIZARD ueberein.

We should check this with W2 and compare against MadEvent?. Either one could be wrong, but this has to be established.

[Juergen Reuter]

Might not even be a defect.

[Juergen Reuter]

Marcus Rammes has reported that there was an issue with the event weights from MadEvent?, so right now we are not sure whether this is a real discrepancy.

[Juergen Reuter]

Marcus Rammes reported that for radiative photons in the process pp->WbWb? + gamma, there is no discrepancy between MadEvent? and WHIZARD.

But still there is the discrepancy between WHIZARD and MadEvent? concerning jet rates. Thanks to Daniel Wiesler for reporting this.

[Juergen Reuter]

new distribution after correctly unweighting MadEvent? events

[Juergen Reuter]

These are some results from 2009 for a MadEvent?/WHIZARD 1 comparison. (version numbers 4.4.8 and 1.92, resp.) The process studied here is u ubar -> b bbar b bbar, all quarks massless, the pure QCD amplitudes. alpha_s = 0.118 fixed in all cases. The cuts are pT(b) > 30 GeV, |eta(b)| < 2.5, Delta R(bb) > 0.4. The cross section is partonic, without PDFs for 14 TeV center-of-mass energy. Here are the results:

WHIZ: 1.2864(97)
      1.2816(22)
      1.2800(15)
      1.2829(14)
MadE: 1.2678(25)
      1.2640(28)
      1.2636(31)

for different random seeds, in fb. So MadEvent? is significantly lower than WHIZARD. Varying the cuts gives the following picture:

pT_b > x                W               M
===================
300             0.5063(11)      0.5058(9)
30              1.2818(14)      1.2678(28)
5               1.8446(68)      1.7051(55)
2               2.1153(76)      1.909(12)
0.5             2.545(11)       2.123(11)

ME does need one order of magnitude more time to reach the same precision for the loosened cuts.

[Juergen Reuter]

W2 file for ME/WH comparison

[Juergen Reuter]

Here are some more data from Daniel Wiesler: I investigated the collinear and soft behaviour of MG/ME 4.4.32 compared to W1 (1.94) and W2 RC3 (recent trunk version) and found rising descrepancies when weakening cuts. To check that these govern the phase space only, I also compared a QED process.

QCD: e-e+ ->
1) uu~
2) uu~g
3) uu~gg

with three different set of cuts (xsec's in fb, alphas fixed at 0.118)

A: pT > 50 GeV, |Eta| < 5 and DeltaR > 1

        ME         ||         W2         ||         W1
1)     684(4)             682,9(2)             683,1(2)
2)     98,2(4)            101,5(5)             101,4(8)
3)     4,28(2)            4,47(8)              4,56(5)

B: pT > 20 GeV, |Eta| < 5 and DeltaR > 0.4

        ME         ||         W2         ||         W1
1)     701(5)            701,6(0,2)            701,2(0,2)
2)     486(4)            436(2)                435(3)
3)     176(1)            141(1)                139(1)

C: pT > 5 GeV, |Eta| < 5 and DeltaR > 0.2

        ME         ||         W2         ||         W1
1)     705(5)           704,4(2)              704,2(1)
2)     1588(7)          1092(4)               1092(6)
3)     2165(12)         1031(10)              1037(8)

The question remains if the discrepancy especially for the cutsets B and C is due to W1/2 or ME? I recommend an additional analysis with e.g. SHERPA for clarification.

[Juergen Reuter]

W1 input file for ME/WH comparison

[Juergen Reuter]

W1 process file for ME/WH comparison

[Juergen Reuter]

ME process file for ME/WH comparison

[dwiesler]

I finally managed to have a closer look into SHERPA (1.2.0) and it's incomplete documentation and found the correct parameters for fixing the value of alphas. Luckily, the results show promising agreement w/ ours, thereby pushing MadEvent? into the dark corner.

BTW: these recent values were produced with the 2.0.0 tarball.

20.04.10
alpha_s = 0.118, sqrts = 500

Cuts                    W2              SH
PT/DR
------------------------------------------------
5/0.2
uu~                     704.43(5)       704(1)
uu~g                    1098(2)         1096(5)
uu~gg                   1038(4)         1026(10)

20/0.4
uu~                     701.25(7)       701(1)
uu~g                    435.2(9)        435(1)
uu~gg                   139.7(6)        139.5(7)

50/1
uu~                     683.69(8)       683.6(9)
uu~g                    101.9(2)        102.0(2)
uu~gg                   4.50(3)         4.52(2)

I'll attach the corresponding run files and try to have another closer look at the MadEvent? figures.

[dwiesler]

w2 comparison file

[dwiesler]

Sherpa run file

[dwiesler]

Another day, another round of benchmarks: These recent MadEvent? figures match the ones from W2 and Sherpa for the two less stringent cut sets. A closer look in the collinear and soft limit shows the deviation from W2/SH to ME.

Sherpa however does not seem to integrate well for the softest cuts and gives a time estimate of 10 days for the uu~gg process, when one promille of accuracy is requested. MadEvent? seems to ignore the required accuracy and simply stops at some point w/ 0.x percent. W2 obviously does a good job here.

Nonetheless: all three generators show a deviation of say three sigmas for the current results, when the integration is repeated. Thus only the top numbers show differing results for W2/SH on one and ME on the other side.

Here, finally, the recent numbers:

21.04.10
alpha_s = 0.118, sqrts = 500

Cuts                    W2              SH              ME
PT/DR
----------------------------------------------------------------
0.5/0.1
uu~                     704.59(1)       705.2(7)        703(2)
uu~g                    2472(2)         2469(3)         2449(6)
uu~gg                   5784(11)        5750(57)        5499(33)

5/0.2
uu~                     704.39(3)       705.0(7)        704(1)
uu~g                    1093.3(1.2)     1096(5)         1098(2)
uu~gg                   1047(3)         1040(5)         1030(2)

20/0.4
uu~                     701.25(7)       701.8(7)        702(2)
uu~g                    435.2(9)        434(1)          435(2)
uu~gg                   139.7(6)        139.5(7)        138.5(4)

[Juergen Reuter]

ME input file for ME/WH comparison

[dwiesler]

Relaxing the two different cuts independently shows a more severe discrepancy for the soft regime (indicated by the ME errors), although the collinear area also seems to have a problem w/ phase space mapping.

2.04.10
alpha_s = 0.118, sqrts = 500

Cuts                    W2              SH              ME
PT/DR
----------------------------------------------------------------

PT fixed to 20 GeV
------------------------
20/0.01
uu~                     701.23(4)       701.8(7)        701(2)
uu~g                    1187(1)         1188(4)         1188(2)
uu~gg                   1179(5)         1171(6)         1097(9)

20/0.05
uu~                     701.23(4)       701.8(7)        701(2)
uu~g                    861(1)          858(2)          861(2)
uu~gg                   599(2)          605(3)          589(1)

20/0.1
uu~                     701.22(4)       701.8(7)        700(2)
uu~g                    719.1(9)        719(2)          719(1)
uu~gg                   416(1)          414(2)          408(1)


DR fixed to 0.4
------------------------
0.05/0.4
uu~                     704.66(2)       705.2(7)        706(2)
uu~g                    2114(2)         2099(10)        2056(8)
uu~gg                   4137(13)        4204(21)        3387(106)

0.1/0.4
uu~                     704.59(1)       705.2(7)        705(2)
uu~g                    1930(1)         1913(10)        1860(6)
uu~gg                   3431(9)         3409(17)        2909(47)

0.5/0.4
uu~                     704.64(1)       705.2(7)        705(2)
uu~g                    1466(1)         1469(7)         1460(4)
uu~gg                   1933(5)         1910(10)        1825(12)

[Juergen Reuter]

Finally the picture seems to be clear that Sherpa and WHIZARD agree even in the collinear and soft regions, while MadEvent? shows considerable deviations. Since the ticket will be accessible even if closed, I close this, as the issue seems solid now.

[Juergen Reuter]

Milestone renamed