whizard is hosted by Hepforge, IPPP Durham

Opened 14 years ago

Closed 14 years ago

Last modified 7 years ago

#175 closed defect (worksforme)

Discrepancy to MadEvent w.r.t. jet rates

Reported by: kilian Owned by: kilian
Priority: P1 Milestone: v42-backlog
Component: core Version: 2.0.0
Severity: critical Keywords:
Cc:

Description

This is the message from Marcus Rammes:

Wenn ich DeltaR (also sqrt(Delta eta2 + Delta phi2)) von den b-Quarks gegen die Photonen plotte, zeigt WHIZARD ein merkwuerdiges Verhalten fuer kollineare Photonen (diff_b_gamma_angle.eps), das MadEvent? nicht aufweist. Haben Sie eine Erklaerung woran das liegen koennte?

Ich habe einen Photon-Cut von E<2 GeV und p_T<2 GeV eingestellt. Saemtliche Cuts und die PDF stimmen bei MG und WHIZARD ueberein.

We should check this with W2 and compare against MadEvent?. Either one could be wrong, but this has to be established.

Attachments (9)

diff_b_gamma_angle.eps (20.0 KB) - added by kilian 14 years ago.
diff_b_bbar_gamma_angle.eps (19.7 KB) - added by Juergen Reuter 14 years ago.
new distribution after correctly unweighting MadEvent? events
comp.sin (540 bytes) - added by Juergen Reuter 14 years ago.
W2 file for ME/WH comparison
whizard.in (859 bytes) - added by Juergen Reuter 14 years ago.
W1 input file for ME/WH comparison
whizard.prc (1.2 KB) - added by Juergen Reuter 14 years ago.
W1 process file for ME/WH comparison
proc_card.dat (3.8 KB) - added by Juergen Reuter 14 years ago.
ME process file for ME/WH comparison
comp.2.sin (595 bytes) - added by dwiesler 14 years ago.
w2 comparison file
Run.dat (731 bytes) - added by dwiesler 14 years ago.
Sherpa run file
run_card.dat (11.4 KB) - added by Juergen Reuter 14 years ago.
ME input file for ME/WH comparison

Download all attachments as: .zip

Change History (21)

Changed 14 years ago by kilian

Attachment: diff_b_gamma_angle.eps added

comment:1 Changed 14 years ago by Juergen Reuter

Milestone: v2.0-rc2v2.0final

Might not even be a defect.

comment:2 Changed 14 years ago by Juergen Reuter

Marcus Rammes has reported that there was an issue with the event weights from MadEvent?, so right now we are not sure whether this is a real discrepancy.

comment:3 Changed 14 years ago by Juergen Reuter

Marcus Rammes reported that for radiative photons in the process pp->WbWb? + gamma, there is no discrepancy between MadEvent? and WHIZARD.

But still there is the discrepancy between WHIZARD and MadEvent? concerning jet rates. Thanks to Daniel Wiesler for reporting this.

Changed 14 years ago by Juergen Reuter

Attachment: diff_b_bbar_gamma_angle.eps added

new distribution after correctly unweighting MadEvent? events

comment:4 Changed 14 years ago by Juergen Reuter

Summary: Discrepancy w.r.t MadEvent for collinear radiationDiscrepancy to MadEvent w.r.t. jet rates

comment:5 Changed 14 years ago by Juergen Reuter

These are some results from 2009 for a MadEvent?/WHIZARD 1 comparison. (version numbers 4.4.8 and 1.92, resp.) The process studied here is u ubar -> b bbar b bbar, all quarks massless, the pure QCD amplitudes. alpha_s = 0.118 fixed in all cases. The cuts are pT(b) > 30 GeV, |eta(b)| < 2.5, Delta R(bb) > 0.4. The cross section is partonic, without PDFs for 14 TeV center-of-mass energy. Here are the results:

WHIZ: 1.2864(97)
      1.2816(22)
      1.2800(15)
      1.2829(14)
MadE: 1.2678(25)
      1.2640(28)
      1.2636(31)

for different random seeds, in fb. So MadEvent? is significantly lower than WHIZARD. Varying the cuts gives the following picture:

pT_b > x                W               M
===================
300             0.5063(11)      0.5058(9)
30              1.2818(14)      1.2678(28)
5               1.8446(68)      1.7051(55)
2               2.1153(76)      1.909(12)
0.5             2.545(11)       2.123(11)

ME does need one order of magnitude more time to reach the same precision for the loosened cuts.

Changed 14 years ago by Juergen Reuter

Attachment: comp.sin added

W2 file for ME/WH comparison

comment:6 Changed 14 years ago by Juergen Reuter

Here are some more data from Daniel Wiesler: I investigated the collinear and soft behaviour of MG/ME 4.4.32 compared to W1 (1.94) and W2 RC3 (recent trunk version) and found rising descrepancies when weakening cuts. To check that these govern the phase space only, I also compared a QED process.

QCD: e-e+ ->
1) uu~
2) uu~g
3) uu~gg

with three different set of cuts (xsec's in fb, alphas fixed at 0.118)

A: pT > 50 GeV, |Eta| < 5 and DeltaR > 1

        ME         ||         W2         ||         W1
1)     684(4)             682,9(2)             683,1(2)
2)     98,2(4)            101,5(5)             101,4(8)
3)     4,28(2)            4,47(8)              4,56(5)

B: pT > 20 GeV, |Eta| < 5 and DeltaR > 0.4

        ME         ||         W2         ||         W1
1)     701(5)            701,6(0,2)            701,2(0,2)
2)     486(4)            436(2)                435(3)
3)     176(1)            141(1)                139(1)

C: pT > 5 GeV, |Eta| < 5 and DeltaR > 0.2

        ME         ||         W2         ||         W1
1)     705(5)           704,4(2)              704,2(1)
2)     1588(7)          1092(4)               1092(6)
3)     2165(12)         1031(10)              1037(8)

The question remains if the discrepancy especially for the cutsets B and C is due to W1/2 or ME? I recommend an additional analysis with e.g. SHERPA for clarification.

Changed 14 years ago by Juergen Reuter

Attachment: whizard.in added

W1 input file for ME/WH comparison

Changed 14 years ago by Juergen Reuter

Attachment: whizard.prc added

W1 process file for ME/WH comparison

Changed 14 years ago by Juergen Reuter

Attachment: proc_card.dat added

ME process file for ME/WH comparison

comment:7 Changed 14 years ago by Juergen Reuter

Milestone: v2.0.0finalgolden-classics

comment:8 Changed 14 years ago by dwiesler

Version: 1.932.0.0

I finally managed to have a closer look into SHERPA (1.2.0) and it's incomplete documentation and found the correct parameters for fixing the value of alphas. Luckily, the results show promising agreement w/ ours, thereby pushing MadEvent? into the dark corner.

BTW: these recent values were produced with the 2.0.0 tarball.

20.04.10
alpha_s = 0.118, sqrts = 500

Cuts                    W2              SH
PT/DR
------------------------------------------------
5/0.2
uu~                     704.43(5)       704(1)
uu~g                    1098(2)         1096(5)
uu~gg                   1038(4)         1026(10)

20/0.4
uu~                     701.25(7)       701(1)
uu~g                    435.2(9)        435(1)
uu~gg                   139.7(6)        139.5(7)

50/1
uu~                     683.69(8)       683.6(9)
uu~g                    101.9(2)        102.0(2)
uu~gg                   4.50(3)         4.52(2)

I'll attach the corresponding run files and try to have another closer look at the MadEvent? figures.

Changed 14 years ago by dwiesler

Attachment: comp.2.sin added

w2 comparison file

Changed 14 years ago by dwiesler

Attachment: Run.dat added

Sherpa run file

comment:9 Changed 14 years ago by dwiesler

Another day, another round of benchmarks: These recent MadEvent? figures match the ones from W2 and Sherpa for the two less stringent cut sets. A closer look in the collinear and soft limit shows the deviation from W2/SH to ME.

Sherpa however does not seem to integrate well for the softest cuts and gives a time estimate of 10 days for the uu~gg process, when one promille of accuracy is requested. MadEvent? seems to ignore the required accuracy and simply stops at some point w/ 0.x percent. W2 obviously does a good job here.

Nonetheless: all three generators show a deviation of say three sigmas for the current results, when the integration is repeated. Thus only the top numbers show differing results for W2/SH on one and ME on the other side.

Here, finally, the recent numbers:

21.04.10
alpha_s = 0.118, sqrts = 500

Cuts                    W2              SH              ME
PT/DR
----------------------------------------------------------------
0.5/0.1
uu~                     704.59(1)       705.2(7)        703(2)
uu~g                    2472(2)         2469(3)         2449(6)
uu~gg                   5784(11)        5750(57)        5499(33)

5/0.2
uu~                     704.39(3)       705.0(7)        704(1)
uu~g                    1093.3(1.2)     1096(5)         1098(2)
uu~gg                   1047(3)         1040(5)         1030(2)

20/0.4
uu~                     701.25(7)       701.8(7)        702(2)
uu~g                    435.2(9)        434(1)          435(2)
uu~gg                   139.7(6)        139.5(7)        138.5(4)

Changed 14 years ago by Juergen Reuter

Attachment: run_card.dat added

ME input file for ME/WH comparison

comment:10 Changed 14 years ago by dwiesler

Relaxing the two different cuts independently shows a more severe discrepancy for the soft regime (indicated by the ME errors), although the collinear area also seems to have a problem w/ phase space mapping.

2.04.10
alpha_s = 0.118, sqrts = 500

Cuts                    W2              SH              ME
PT/DR
----------------------------------------------------------------

PT fixed to 20 GeV
------------------------
20/0.01
uu~                     701.23(4)       701.8(7)        701(2)
uu~g                    1187(1)         1188(4)         1188(2)
uu~gg                   1179(5)         1171(6)         1097(9)

20/0.05
uu~                     701.23(4)       701.8(7)        701(2)
uu~g                    861(1)          858(2)          861(2)
uu~gg                   599(2)          605(3)          589(1)

20/0.1
uu~                     701.22(4)       701.8(7)        700(2)
uu~g                    719.1(9)        719(2)          719(1)
uu~gg                   416(1)          414(2)          408(1)


DR fixed to 0.4
------------------------
0.05/0.4
uu~                     704.66(2)       705.2(7)        706(2)
uu~g                    2114(2)         2099(10)        2056(8)
uu~gg                   4137(13)        4204(21)        3387(106)

0.1/0.4
uu~                     704.59(1)       705.2(7)        705(2)
uu~g                    1930(1)         1913(10)        1860(6)
uu~gg                   3431(9)         3409(17)        2909(47)

0.5/0.4
uu~                     704.64(1)       705.2(7)        705(2)
uu~g                    1466(1)         1469(7)         1460(4)
uu~gg                   1933(5)         1910(10)        1825(12)

comment:11 Changed 14 years ago by Juergen Reuter

Resolution: worksforme
Status: newclosed

Finally the picture seems to be clear that Sherpa and WHIZARD agree even in the collinear and soft regions, while MadEvent? shows considerable deviations. Since the ticket will be accessible even if closed, I close this, as the issue seems solid now.

comment:12 Changed 7 years ago by Juergen Reuter

Milestone: golden-classicsv42-backlog

Milestone renamed

Note: See TracTickets for help on using tickets.