Phase space / integration efficiency in WHIZARD 2 (?)

[Juergen Reuter]

PS efficiency: W2 seems to need considerably more statistics to achieve the same

(absolute and relative) uncertainty as W1: Example: e1, E1 -> n1, N1, H, 7 iterat., 50,000 calls each: W1: 84.85(0.16) fb W2: 85.09(1.31) fb These are the corresponding runs: me -> 0.0000000000000000 mH -> 115.00000000000000 sqrts = 500.00000000000000 Beam data (collision):

e- (mass = 0.0000000 GeV) e+ (mass = 0.0000000 GeV) sqrts = 500.0000000000000 GeV

| Integrating process 'nnh' |=============================================================================| | It Calls Integral[fb] Error[fb] Err[%] Acc Eff[%] Chi2 N[It] | |=============================================================================|

1 50000 8.3899236E+01 4.17E+00 4.97 11.10* 0.30 2 50000 9.0903348E+01 3.74E+00 4.11 9.20* 0.33 3 50000 8.3494054E+01 3.23E+00 3.86 8.64* 0.34 4 50000 8.7722228E+01 3.42E+00 3.90 8.72 0.31 5 50000 8.1941515E+01 3.22E+00 3.93 8.78 0.32 6 50000 8.4430988E+01 3.36E+00 3.98 8.90 0.31 7 50000 8.4497482E+01 3.32E+00 3.93 8.80* 0.34

|=============================================================================|

7 350000 8.5095091E+01 1.31E+00 1.53 9.08 0.34 0.73 7

|============================================================================= * * *

!----------------------------------------------------------------------------- ! Adapting (variable wgts.): 6 samples of 50000 calls ... 2 50000 8.4581446E+01 1.83E+00 2.16 4.83 2.07 3 50000 8.4825351E+01 3.23E-01 0.38 0.85* 19.60 4 50000 8.5295714E+01 2.72E-01 0.32 0.71* 5.87 5 50000 8.5144108E+01 2.69E-01 0.32 0.71* 6.41 6 50000 8.5198347E+01 2.77E-01 0.33 0.73 4.96 7 50000 8.4383089E+01 2.72E-01 0.32 0.72 5.88 !----------------------------------------------------------------------------- ! Integrating (fixed wgts.): 3 samples of 50000 calls ...

8 150000 8.4855400E+01 1.59E-01 0.19 0.73 4.47 0.18 3

!-----------------------------------------------------------------------------

[kilian]

Yes, W1 and W2 algorithms for phase space are similar, but not identical. Giving W2 the W1 phase space file works and should give the expected result. The W1 phase space code is incomprehensible with its special cases, etc. It has to be studied whether this is generic, and what is the best solution that keeps simplicity.

The issue is important, but this is a typical beta testing improvement, I'd say ...

[Juergen Reuter]

Should be checked in the beta testing phase. Moving this to the next milestone. But keeping the highest priority!!!

[Juergen Reuter]

New formatting: PS efficiency: W2 seems to need considerably more statistics to achieve the same (absolute and relative) uncertainty as W1: Example: e1, E1 -> n1, N1, H, 7 iterat., 50,000 calls each: W1: 84.85(0.16) fb W2: 85.09(1.31) fb These are the corresponding runs:

me -> 0.0000000000000000 
mH -> 115.00000000000000 
sqrts = 500.00000000000000 
Beam data (collision):
  e- (mass = 0.0000000 GeV) 
  e+ (mass = 0.0000000 GeV) 
  sqrts = 500.0000000000000 GeV
| Integrating process 'nnh' 
|=============================================================================| 
| It Calls Integral[fb] Error[fb] Err[%] Acc Eff[%] Chi2 N[It]                | 
|=============================================================================|
  1 50000 8.3899236E+01 4.17E+00 4.97 11.10* 0.30 
  2 50000 9.0903348E+01 3.74E+00 4.11  9.20* 0.33 
  3 50000 8.3494054E+01 3.23E+00 3.86  8.64* 0.34 
  4 50000 8.7722228E+01 3.42E+00 3.90  8.72  0.31 
  5 50000 8.1941515E+01 3.22E+00 3.93  8.78  0.32 
  6 50000 8.4430988E+01 3.36E+00 3.98  8.90  0.31 
  7 50000 8.4497482E+01 3.32E+00 3.93  8.80* 0.34
|=============================================================================
| 7 350000 8.5095091E+01 1.31E+00 1.53 9.08  0.34 0.73 7
|============================================================================= 
*** *** ***
!----------------------------------------------------------------------------- 
! Adapting (variable wgts.): 6 samples of 50000 calls ... 
  2 50000 8.4581446E+01 1.83E+00 2.16 4.83   2.07 
  3 50000 8.4825351E+01 3.23E-01 0.38 0.85* 19.60 
  4 50000 8.5295714E+01 2.72E-01 0.32 0.71*  5.87 
  5 50000 8.5144108E+01 2.69E-01 0.32 0.71*  6.41 
  6 50000 8.5198347E+01 2.77E-01 0.33 0.73   4.96 
  7 50000 8.4383089E+01 2.72E-01 0.32 0.72   5.88 !----------------------------------------------------------------------------- 
! Integrating (fixed wgts.): 3 samples of 50000 calls ...
  8 150000 8.4855400E+01 1.59E-01 0.19 0.73  4.47 0.18 3
!-----------------------------------------------------------------------------

[Juergen Reuter]

File for which WHIZARD 2 does not find phase space channels.

[Juergen Reuter]

There is another problem (maybe it's the same, I don't know yet). For e+ e- -> t W- bbar W2 does not find a valid phase space channel. I have no clue why. Both e+ e- -> t ttbar and tbar -> W- bbar work actually!!! I attached the input file "topproblem.sin".

[Juergen Reuter]

Test file for the eewwbb process.

[Juergen Reuter]

On the other hand, the process "e+ e- -> Wp, Wm, b, bbar" is working (OK, it does not only go via top pair production). The integration is worse for WHIZARD 2, not remarkably, but definitely. Here is the W1 run:

 ! Preparing (fixed weights):   1 sample of      20000 calls ...
    1      20000  5.4703363E+02  9.43E+00    1.72    2.44*  6.74    0.00   1
 !-----------------------------------------------------------------------------
 ! Adapting (variable wgts.):  10 samples of      20000 calls ...
    2      20000  5.3526895E+02  9.11E+00    1.70    2.41*  8.89
    3      20000  5.3634931E+02  2.76E+00    0.51    0.73* 23.59
    4      20000  5.3287896E+02  2.00E+00    0.37    0.53* 28.37
    5      20000  5.3050484E+02  1.81E+00    0.34    0.48* 27.76
    6      20000  5.3137846E+02  1.78E+00    0.34    0.47* 15.05
    7      20000  5.3307291E+02  1.82E+00    0.34    0.48  19.13
    8      20000  5.3121154E+02  1.84E+00    0.35    0.49  22.55
    9      20000  5.3310260E+02  1.90E+00    0.36    0.50  19.70
   10      20000  5.3017162E+02  1.86E+00    0.35    0.50  25.22
   11      20000  5.2951544E+02  1.92E+00    0.36    0.51  18.59
 !-----------------------------------------------------------------------------
 ! Integrating (fixed wgts.):   3 samples of      20000 calls ...
   12      60000  5.3143616E+02  1.02E+00    0.19    0.47* 19.01    0.37   3
 !-----------------------------------------------------------------------------
 !
 ! Time estimate for generating 10000 unweighted events:    0h 00m 28s
 !=============================================================================
 ! Summary (all processes):
 !-----------------------------------------------------------------------------
 ! Process ID     Integral[fb]  Error[fb]   Err[%]        Frac[%]
 !-----------------------------------------------------------------------------
   eewwbb         5.3143616E+02  1.02E+00    0.19         100.00
 !-----------------------------------------------------------------------------
   sum            5.3143616E+02  1.02E+00    0.19         100.00
 !=============================================================================

And the corresponding W2 with the same number of calls per iterations:

  | Integrating process 'eewwbb'
|=============================================================================|
| It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It] |
|=============================================================================|
   1      20000  5.2173904E+02  4.90E+00    0.94    1.33*   0.77
   2      20000  5.1463516E+02  3.47E+00    0.67    0.95*   1.48
   3      20000  5.1748960E+02  4.75E+00    0.92    1.30    0.88
   4      20000  5.2171187E+02  4.38E+00    0.84    1.19*   1.15
   5      20000  5.1759271E+02  3.27E+00    0.63    0.89*   2.13
   6      20000  5.2917309E+02  5.94E+00    1.12    1.59    0.67
   7      20000  5.3086201E+02  5.44E+00    1.03    1.45*   1.21
   8      20000  5.1846538E+02  3.96E+00    0.76    1.08*   1.56
   9      20000  5.1908832E+02  3.48E+00    0.67    0.95*   1.94
  10      20000  5.2341596E+02  4.84E+00    0.92    1.31    1.01
  11      20000  5.2026043E+02  3.36E+00    0.65    0.91*   2.08
|=============================================================================|
  11     220000  5.2000457E+02  1.24E+00    0.24    1.11    2.08    1.06  11
|=============================================================================|
|=========================      Results Summary      =========================|
 eewwbb:  5.20004574E+02 +- 1.24E+00 fb
|=============================================================================|
| WHIZARD run finished.
|=============================================================================|

I attached the input file for it, eewwbb.sin. Also, WHIZARD 2 seems to be slower than WHIZARD 1, even with gfortran, which also uses optimization!!! WK, I also DON'T GET the stars for the improvement???

[kilian]

There were two bugs in the phase space module which caused many (most) important s-channel cascades to be missed. Fixed in [1418].

For the peculiar ee -> tBW process, W2 now generates phase space. With default parameters (phs_off_shell = 1), it generates 1 channel, the resonant one. With phs_off_shell = 2 in the input script, the subdominant contributions are included, and the reweighting efficiency (but not the accuracy) becomes much better. In most cases, however, phs_off_shell = 1 should be sufficient, so I do not change the default, which was also the W1 default.

If JR confirms that W2 works now as expected also for complicated processes, we can close this ticket.

Note: W2 phase space is not identical to W1, but should be similar. If the remaining differences actually matter, I should follow the W1 algorithm even more closely.

[Juergen Reuter]

W1 Test1 input file

[Juergen Reuter]

W2 Test1 input file

[Juergen Reuter]

Here is some test for the performance: W1:

 ! Preparing (fixed weights):   1 sample of     100000 calls ...
    1     100000  3.2263360E+01  1.67E+00    5.19   16.40*  2.07    0.00   1
 !-----------------------------------------------------------------------------
 ! Adapting (variable wgts.):  20 samples of     100000 calls ...
    2     100000  3.4604047E+01  2.18E+00    6.29   19.90   1.39
    3     100000  3.2070428E+01  2.51E-01    0.78    2.48*  2.66
    4     100000  3.2464654E+01  1.53E-01    0.47    1.49*  3.56
    5     100000  3.2691531E+01  2.32E-01    0.71    2.24   2.31
    6     100000  3.2632754E+01  2.66E-01    0.82    2.58   1.69
    7     100000  3.2483852E+01  1.84E-01    0.57    1.79   1.53
    8     100000  3.3558338E+01  9.65E-01    2.88    9.09   0.45
    9     100000  3.2865211E+01  1.90E-01    0.58    1.83   1.95
   10     100000  3.2435646E+01  1.71E-01    0.53    1.67   2.43
   11     100000  3.2972772E+01  1.69E-01    0.51    1.62   2.77
   12     100000  3.2488525E+01  1.52E-01    0.47    1.48*  4.30
   13     100000  3.2517551E+01  1.72E-01    0.53    1.67   3.13
   14     100000  3.2406127E+01  1.51E-01    0.47    1.48*  2.90
   15     100000  3.2645175E+01  1.50E-01    0.46    1.45*  4.17
   16     100000  3.2412268E+01  1.52E-01    0.47    1.48   3.46
   17     100000  3.2509009E+01  1.52E-01    0.47    1.48   4.10
   18     100000  3.2688375E+01  1.60E-01    0.49    1.54   3.69
   19     100000  3.2648697E+01  2.08E-01    0.64    2.01   2.41
   20     100000  3.5570866E+01  3.23E+00    9.08   28.70   0.24
   21     100000  3.2649440E+01  1.98E-01    0.61    1.92   3.24
 !-----------------------------------------------------------------------------
 ! Integrating (fixed wgts.):   3 samples of     100000 calls ...
   22     300000  3.2776644E+01  1.07E-01    0.33    1.79   2.17    2.28   3
 !-----------------------------------------------------------------------------
 !
 ! Time estimate for generating 10000 unweighted events:    0h 03m 47s
 !=============================================================================
 ! Summary (all processes):
 !-----------------------------------------------------------------------------
 ! Process ID     Integral[fb]  Error[fb]   Err[%]        Frac[%]
 !-----------------------------------------------------------------------------
   top_bbuddu     3.2776644E+01  1.07E-01    0.33         100.00
 !-----------------------------------------------------------------------------
   sum            3.2776644E+01  1.07E-01    0.33         100.00
 !=============================================================================

and W2 (same number of calls)

| Integrating process 'sm_proc_01'
| Generating phase space, writing file 'sm_proc_01.phs' (this may take a while)
| Found 138 phase space channels.
| Applying user-defined cuts.
| Using partonic energy as event scale.
| iterations = 1:100000, 20:100000, 1:1000000
| Creating grids: 100000 calls, 14 dimensions, 20 bins, stratified = T
|=============================================================================|
| It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It] |
|=============================================================================|
   1     100000  3.2729756E+01  8.11E-01    2.48    7.83*   4.37
|-----------------------------------------------------------------------------|
   1     100000  3.2729756E+01  8.11E-01    2.48    7.83    4.37
|-----------------------------------------------------------------------------|
   2     100000  3.2011826E+01  9.12E-01    2.85    9.01    2.68
   3     100000  3.2037508E+01  5.39E-01    1.68    5.32*   2.09
   4     100000  3.2183103E+01  3.76E-01    1.17    3.69*   6.33
   5     100000  3.2092805E+01  3.22E-01    1.00    3.17*   7.81
   6     100000  3.2049451E+01  3.73E-01    1.16    3.68    4.10

   7     100000  3.2357153E+01  3.05E-01    0.94    2.98*   5.30
   8     100000  3.3769060E+01  1.61E+00    4.78   15.11    1.53
   9     100000  3.2310114E+01  3.26E-01    1.01    3.19*   5.34
  10     100000  3.2318897E+01  2.96E-01    0.91    2.89*   4.66
  11     100000  3.1909523E+01  2.60E-01    0.81    2.57*   5.82
  12     100000  3.1941737E+01  2.54E-01    0.79    2.51*   5.27
  13     100000  3.2218696E+01  2.76E-01    0.86    2.71    4.92
  14     100000  3.1721788E+01  2.26E-01    0.71    2.25*   6.13
  15     100000  3.2135703E+01  2.20E-01    0.69    2.17*   5.97
  16     100000  3.2750539E+01  2.66E-01    0.81    2.57    4.13
  17     100000  3.2177072E+01  2.18E-01    0.68    2.15*   6.25
  18     100000  3.2463125E+01  2.28E-01    0.70    2.22    5.85
  19     100000  3.2339236E+01  2.14E-01    0.66    2.09*   5.85
  20     100000  3.1952483E+01  2.05E-01    0.64    2.03*   6.62
  21     100000  3.2405863E+01  1.97E-01    0.61    1.92*   6.38
|-----------------------------------------------------------------------------|
  21    2000000  3.2191632E+01  6.08E-02    0.19    2.67    6.38    0.91  20
|-----------------------------------------------------------------------------|
  22    1000000  3.2692888E+01  2.42E-01    0.74    7.41    0.94
|=============================================================================|
  22    1000000  3.2692888E+01  2.42E-01    0.74    7.41    0.94
|=============================================================================|
|=========================      Results Summary      =========================|
 sm_proc_01:  3.26928884E+01 +- 2.42E-01 fb
|=============================================================================|
| WHIZARD run finished.
|=============================================================================|

One sees that with less than a third of statistics W1 performs better by more than a factor of 2 !!! I attached the files "w1_test1.in" and "w2_test2.sin".

[kilian]

Well, it depends. On the other hand, the W1 output shows less stability of the result. How many PHS channels did W1 generate for this? It may as well be that the quoted W1 error is underestimated. Note the ratio of the chi2 values. Both are acceptable, but to me it looks like the results are rather comparable. What happens if I do the last W2 iteration with 300k as W1? (Takes some time to reproduce, of course.)

Note that it is advisable to use similar call numbers in adaptation and final integration. Maybe check again with iterations = 10:100000, 5:100000

[Juergen Reuter]

Here is the 500 GeV case for the same process (e+ e- -> b bbar u dbar d ubar), the alphas value can be read off from the smext.sin file. What puzzles me is the behavior of W2 for the choice of iterations, namely from the entry iterations = 1:100000, 20:100000, 1:300000 it takes 1 sample of 100,000 calls, then 20 samples of 100,000, then one of 20x100,000 = 2,000,000 calls, and then a single one of 300,000. One should explain this arithmetics. W1 result:

 ! Preparing (fixed weights):   1 sample of     100000 calls ...
    1     100000  4.5364750E+01  1.65E+00    3.65   11.53*  2.18    0.00   1
 !-----------------------------------------------------------------------------
 ! Adapting (variable wgts.):  20 samples of     100000 calls ...
    2     100000  5.1295910E+01  2.22E+00    4.32   13.67   1.87
    3     100000  5.1084889E+01  1.13E+00    2.22    7.02*  0.40
    4     100000  4.9276507E+01  2.59E-01    0.53    1.66*  1.31
    5     100000  4.9767800E+01  2.11E-01    0.42    1.34*  2.19
    6     100000  4.9911449E+01  2.21E-01    0.44    1.40   1.16
    7     100000  4.9759561E+01  2.32E-01    0.47    1.48   2.76
    8     100000  4.9298922E+01  1.69E-01    0.34    1.09*  2.11
    9     100000  4.9884048E+01  1.65E-01    0.33    1.05*  3.75
   10     100000  5.0363873E+01  8.00E-01    1.59    5.03   1.11
   11     100000  4.9730273E+01  1.90E-01    0.38    1.21   2.73
   12     100000  4.9756725E+01  1.95E-01    0.39    1.24   3.33
   13     100000  4.9608139E+01  1.72E-01    0.35    1.10   3.61
   14     100000  5.0244454E+01  1.89E-01    0.38    1.19   3.08
   15     100000  4.9833243E+01  1.73E-01    0.35    1.10   4.12
   16     100000  5.0362088E+01  3.38E-01    0.67    2.12   1.05
   17     100000  5.2830965E+01  3.09E+00    5.85   18.49   0.15
   18     100000  4.9689442E+01  2.39E-01    0.48    1.52   2.59
   19     100000  4.9724352E+01  1.89E-01    0.38    1.20   3.82
   20     100000  4.9835579E+01  1.83E-01    0.37    1.16   2.55
   21     100000  5.0674960E+01  3.47E-01    0.68    2.16   1.72
 !-----------------------------------------------------------------------------
 ! Integrating (fixed wgts.):   3 samples of     100000 calls ...
   22     300000  4.9669421E+01  3.19E-01    0.64    3.52   0.13    1.30   3
 !-----------------------------------------------------------------------------
 !
 ! Time estimate for generating 10000 unweighted events:    0h 05m 24s
 !=============================================================================
 ! Summary (all processes):
 !-----------------------------------------------------------------------------
 ! Process ID     Integral[fb]  Error[fb]   Err[%]        Frac[%]
 !-----------------------------------------------------------------------------
   top_bbuddu     4.9669421E+01  3.19E-01    0.64         100.00
 !-----------------------------------------------------------------------------
   sum            4.9669421E+01  3.19E-01    0.64         100.00
 !=============================================================================

The W2 result:

|=============================================================================|
| It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It] |
|=============================================================================|
   1     100000  4.9779957E+01  1.36E+00    2.73    8.64*   3.19
|-----------------------------------------------------------------------------|
   1     100000  4.9779957E+01  1.36E+00    2.73    8.64    3.19
|-----------------------------------------------------------------------------|
   2     100000  4.9051720E+01  1.15E+00    2.35    7.42*   4.79
   3     100000  5.0131205E+01  7.47E-01    1.49    4.71*   3.07
   4     100000  5.0647475E+01  6.94E-01    1.37    4.33*   5.79
   5     100000  4.9510900E+01  5.25E-01    1.06    3.35*   8.85
   6     100000  4.9852742E+01  5.01E-01    1.00    3.18*   7.02
   7     100000  4.8924123E+01  4.40E-01    0.90    2.84*   8.07
   8     100000  5.0707278E+01  4.52E-01    0.89    2.82*   6.97
   9     100000  5.1300058E+01  1.56E+00    3.04    9.63    2.71
  10     100000  4.9644092E+01  4.51E-01    0.91    2.87*   5.42
  11     100000  4.9945167E+01  3.97E-01    0.79    2.51*   6.59
  12     100000  4.9543773E+01  3.87E-01    0.78    2.47*   5.30
  13     100000  5.0482730E+01  4.70E-01    0.93    2.94    3.59
  14     100000  4.9918112E+01  3.55E-01    0.71    2.25*   6.42
  15     100000  4.9407675E+01  3.37E-01    0.68    2.16*   7.17
  16     100000  5.0440548E+01  8.69E-01    1.72    5.45    5.14
  17     100000  5.0616984E+01  3.41E-01    0.67    2.13*   7.46
  18     100000  4.9746183E+01  3.23E-01    0.65    2.06*   7.91
  19     100000  5.8820099E+01  9.10E+00   15.47   48.91    0.23
  20     100000  5.0280312E+01  6.89E-01    1.37    4.33*   5.24
  21     100000  4.9915886E+01  3.19E-01    0.64    2.02*   6.28
|-----------------------------------------------------------------------------|
  21    2000000  4.9908547E+01  1.03E-01    0.21    2.93    6.28    1.18  20
|-----------------------------------------------------------------------------|
  22     300000  5.0021995E+01  2.65E-01    0.53    2.91    4.13
|=============================================================================|
  22     300000  5.0021995E+01  2.65E-01    0.53    2.91    4.13
|=============================================================================|
|=========================      Results Summary      =========================|
 sm_proc_01:  5.00219952E+01 +- 2.65E-01 fb
|=============================================================================|
| WHIZARD run finished.
|=============================================================================|

[kilian]

Looks better ...

Default iterations: these values are a bit odd, indeed. For 2->6, I reduced the number of warmup iterations to 15 and increase the number of integration iterations to 5. The warmup phase has to be long in order to catch difficult corners of phase space. See [1425].

The default iteration setting is in commands.f90 in procedure iterations_lists_init_default.

[Juergen Reuter]

Ok, for the next process, e+ e- -> b bbar u uubar g g (with the same cuts), W2 behaves completely erratic: Here is the result from W1 (which is also not a brilliant integration):

 ! Preparing (fixed weights):   1 sample of     100000 calls ...
    1     100000  7.1565766E+00  6.97E-01    9.74   30.79*  0.28    0.00   1
 !-----------------------------------------------------------------------------
 ! Adapting (variable wgts.):  20 samples of     100000 calls ...
    2     100000  9.3561876E+00  9.65E-01   10.31   32.61   0.30
    3     100000  7.8659815E+00  4.21E-01    5.35   16.92*  0.34
    4     100000  7.5611881E+00  2.77E-01    3.67   11.60*  0.36
    5     100000  8.3889572E+00  3.94E-01    4.69   14.84   0.33
    6     100000  7.9739481E+00  3.39E-01    4.25   13.42   0.53
    7     100000  8.1203065E+00  2.74E-01    3.38   10.68*  0.44
    8     100000  7.8237152E+00  1.96E-01    2.50    7.92*  0.58
    9     100000  8.0345080E+00  1.98E-01    2.47    7.81*  0.56
   10     100000  8.6685701E+00  8.45E-01    9.74   30.82   0.27
   11     100000  8.0069320E+00  3.56E-01    4.45   14.06   0.36
   12     100000  8.2053636E+00  3.14E-01    3.82   12.09   0.35
   13     100000  1.4866747E+01  7.14E+00   48.05  151.93   0.10
   14     100000  8.4744383E+00  4.00E-01    4.72   14.93   0.45
   15     100000  7.7181152E+00  1.99E-01    2.57    8.14   0.53
   16     100000  8.0374398E+00  2.26E-01    2.82    8.91   0.54
   17     100000  8.0848713E+00  1.74E-01    2.16    6.83*  0.67
   18     100000  8.0052281E+00  1.78E-01    2.23    7.05   0.71
   19     100000  8.6641383E+00  3.57E-01    4.12   13.02   0.53
   20     100000  8.0701276E+00  1.94E-01    2.41    7.61   0.70
   21     100000  8.2027817E+00  1.65E-01    2.01    6.36*  0.77
 !-----------------------------------------------------------------------------
 ! Integrating (fixed wgts.):   3 samples of     100000 calls ...
   22     300000  8.4622174E+00  1.11E-01    1.32    7.21   0.47    0.63   3
 !-----------------------------------------------------------------------------
 !
 ! Time estimate for generating 10000 unweighted events:    1h 11m 54s
 !=============================================================================
 ! Summary (all processes):
 !-----------------------------------------------------------------------------
 ! Process ID     Integral[fb]  Error[fb]   Err[%]        Frac[%]
 !-----------------------------------------------------------------------------
   top_bbuugg     8.4622174E+00  1.11E-01    1.32         100.00
 !-----------------------------------------------------------------------------
   sum            8.4622174E+00  1.11E-01    1.32         100.00
 !=============================================================================

Here, the W2 result:

|=============================================================================|
| It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It] |
|=============================================================================|
   1     100000  3.1676968E-04  1.80E-04   56.69  179.27*   0.04
|-----------------------------------------------------------------------------|
   1     100000  3.1676968E-04  1.80E-04   56.69  179.27    0.04
|-----------------------------------------------------------------------------|
^[[6~^[[5~   2     100000  2.2769966E-04  1.63E-04   71.79  227.01    0.04
   3     100000  2.8529960E-05  1.13E-05   39.59  125.19*   0.00
   4     100000  7.4744268E-04  6.29E-04   84.09  265.91    0.03
   5     100000  3.3735474E-04  1.35E-04   40.02  126.56*   0.05
   6     100000  2.3806180E-02  1.51E-02   63.25  200.02    0.02
   7     100000  5.1933941E-02  3.02E-02   58.17  183.95*   0.01
   8     100000  1.3256384E-01  4.73E-02   35.68  112.83*   0.04
   9     100000  6.4297582E-01  2.28E-01   35.43  112.05*   0.03
  10     100000  3.1453714E+00  1.92E+00   61.00  192.90    0.01
  11     100000  6.3764618E-01  1.83E-01   28.77   90.98*   0.01
  12     100000  1.0825902E+00  2.94E-01   27.19   85.98*   0.02
  13     100000  1.3043833E+00  3.41E-01   26.15   82.69*   0.02
  14     100000  7.0256897E-01  9.55E-02   13.60   43.00*   0.05
  15     100000  3.1588838E+00  6.87E-01   21.75   68.77    0.02
  16     100000  6.5918253E+00  3.28E+00   49.82  157.53    0.01
  17     100000  1.5413305E+00  3.84E-01   24.94   78.87*   0.02
  18     100000  1.1551475E+01  7.48E+00   64.74  204.74    0.01
  19     100000  4.8870708E+00  2.23E+00   45.64  144.33*   0.01
  20     100000  4.7898482E+00  1.72E+00   35.96  113.71*   0.02
  21     100000  3.4052558E+00  7.22E-01   21.21   67.08*   0.02
|-----------------------------------------------------------------------------|
  21    2000000  3.1882567E-05  1.12E-05   35.21  498.01    0.02   10.77  20
|-----------------------------------------------------------------------------|
  22     300000  5.1060126E+00  1.92E+00   37.63  206.09    0.01
|=============================================================================|
  22     300000  5.1060126E+00  1.92E+00   37.63  206.09    0.01
|=============================================================================|
|=========================      Results Summary      =========================|
 sm_proc_02:  5.10601263E+00 +- 1.92E+00 fb
|=============================================================================|

That is devastating.

[Christian Speckner]

I can confirm this with an example as simple as e+,e- -> W+W-. The command file

model = SM process eeww = "e+", "e-" -> "W+", "W-" compile scan sqrts (200 -> 1000 /+ 100) {

integrate (eeww) {iterations = 5:5000}

} show (results)

reveals that, while the relative error is <1% for 200GeV, it increases steadily with CMS energy and reaches about 10% for 1TeV. For the same process, WHIZARD 1.94 remains stable over the whole energy range with the error being always <1% as should be expected for this simple process. I'm attaching the W2 output log and the W1 log for 1TeV.

[Christian Speckner]

Shit, didn't preserve my formatting in the script. Anyways, you get the idea :)

[Christian Speckner]

It seems that applying \cos\theta cuts like

model = SM
process eeww = "e+", "e-" -> "W+", "W-"
compile
cuts = all -0.9 < cos(Theta) < 0.9 ["W+":"W-"]
scan sqrts (200 -> 1000 /+ 100) {
        integrate (eeww) {iterations = 5:5000}
}
show (results)

stabilizes the integration (see the attached output), so I would guess that something is not quite right with the t- resp. u-channel mappings.

[Christian Speckner]

Oh, before going finally to bed: I should mention perhaps that, if no cuts are applied, increasing the number of calls is not effective in lowering the error, at least definitely not like \sqrt{n}.

[kilian]

@cspeckn: Thanks for this example, I already found a discrepancy W1/2. Investigating further ...

[kilian]

I found two places where the W boson was treated as massive, but should be treated as massless. (There is a threshold mass for this which is 100 GeV by default). Changing this [1563], Christian's example behaves now as it should. I could not yet check the 6-particle process.

[kilian]

OK, the process ee -> b bbar u ubar g g fails with W2 due to a weird reason: The W2 SM model file contains the Hgg vertex. W2 uses this for phase-space generation, i.e., the on-shell decay H -> gg. Formally, this is the leading contribution. In practice, it is a tiny contribution. Since the Higgs is extremely narrow, W2 is not able to adapt to the bulk of the phase space.

I have to find out why W1 did not run into this problem, and how to avoid it.

[kilian]

Here is the new W2 result for the above sm_proc01:

 Integrating process 'sm_proc_01'
| Generating phase space, writing file 'sm_proc_01.phs' (this may take a while)
| Found 352 phase space channels.
| Applying user-defined cuts.
| Using partonic energy as event scale.
| iterations = 1:100000, 20:100000, 1:1000000
| Creating grids
| 100000 calls, 352 channels, 14 dimensions, 20 bins, stratified = T
|=============================================================================|
| It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It] |
|=============================================================================|
   1     100000  3.2109281E+01  1.23E+00    3.83   12.13*   4.33
|-----------------------------------------------------------------------------|
   1     100000  3.2109281E+01  1.23E+00    3.83   12.13    4.33
|-----------------------------------------------------------------------------|
   2     100000  3.2027060E+01  1.20E+00    3.74   11.83*   4.50
   3     100000  3.3189439E+01  7.52E-01    2.27    7.16*   3.45
   4     100000  3.2680151E+01  6.27E-01    1.92    6.07*   4.38
   5     100000  3.1832305E+01  3.85E-01    1.21    3.83*   6.93
   6     100000  3.1259321E+01  3.51E-01    1.12    3.55*   8.23
   7     100000  3.1815559E+01  3.94E-01    1.24    3.91    5.49
   8     100000  3.1635421E+01  3.76E-01    1.19    3.76*   5.74
   9     100000  3.2087882E+01  5.91E-01    1.84    5.82    4.13
  10     100000  3.1732602E+01  3.09E-01    0.97    3.08*   7.56
  11     100000  3.2091797E+01  3.12E-01    0.97    3.08*   6.92
  12     100000  3.2829916E+01  5.04E-01    1.54    4.86    2.87
  13     100000  3.2494444E+01  6.70E-01    2.06    6.52    1.85
  14     100000  3.2320154E+01  3.20E-01    0.99    3.13*   3.29
  15     100000  3.2075184E+01  2.83E-01    0.88    2.79*   4.03
  16     100000  3.1606027E+01  2.51E-01    0.80    2.52*   6.35
  17     100000  3.1949282E+01  2.62E-01    0.82    2.59    3.99
  18     100000  3.2041332E+01  2.57E-01    0.80    2.54*   5.95
  19     100000  3.2135719E+01  2.41E-01    0.75    2.38*   5.23
  20     100000  3.2308232E+01  2.25E-01    0.70    2.21*   6.46
  21     100000  3.2242980E+01  2.22E-01    0.69    2.18*   5.64
|-----------------------------------------------------------------------------|
  21    2000000  3.2035792E+01  7.29E-02    0.23    3.22    5.64    1.07  20
|-----------------------------------------------------------------------------|
  22    1000000  3.2082018E+01  6.78E-02    0.21    2.11*   3.07
|=============================================================================|
  22    1000000  3.2082018E+01  6.78E-02    0.21    2.11    3.07
|=============================================================================|

This is even better than the quoted W1 result.

[kilian]

One step further: with r1606, collinear and IR mappings are enabled. This may solve the problem. Checking the above processes ...

[kilian]

So far, everything is fine with e+ e- -> bbg, bbgg, bbuu, bbuug. These are already nontrivial tests.

The W2 version I'm testing this with has the Hgg coupling switched off. This may become the default, with an option to switch it on at runtime. In W1, the Hgg channels are typically dropped during adaption.

With the current setup, W2 generates more channels than W1. The integration error is similar for equal parameters, but the reweighting efficiency has a tendency to be better in the end (e.g., 8% vs. 5%). This may be worthwhile some slowdown in the integration due to the extra channels.

I still have to check e+ e- -> bbuugg with W2 against W1.

[kilian]

This time W2 wins over W1. Here is the W1 output for e+ e- -> b bbar u ubar g g (360 GeV)

! WHIZARD run for process bbuugg:
 !=============================================================================
 ! Input checksum = 4AEBD346C326C0C0D72E857562687483
 !-----------------------------------------------------------------------------
 ! It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It]
 !-----------------------------------------------------------------------------
    1     200000  9.0832530E+00  5.94E-01    6.54   29.25*  0.18    0.00   1
 !-----------------------------------------------------------------------------
    2     200000  7.8578527E+00  5.98E-01    7.61   34.05   0.19
    3     200000  9.6514525E+00  3.17E-01    3.29   14.71*  0.35
    4     200000  2.4021787E+01  1.40E+01   58.33  260.86   0.08
    5     200000  9.4983669E+00  2.17E-01    2.28   10.21*  0.45
    6     200000  9.6090907E+00  2.13E-01    2.22    9.92*  0.37
    7     200000  9.3608235E+00  1.28E-01    1.36    6.10*  0.63
    8     200000  1.1261227E+01  1.07E+00    9.51   42.55   0.16
    9     200000  1.0296078E+01  2.75E-01    2.67   11.95   0.38
   10     200000  1.2077121E+01  1.62E+00   13.45   60.14   0.22
   11     200000  1.0525447E+01  1.57E-01    1.49    6.68   0.47
 !-----------------------------------------------------------------------------
   12     600000  1.0429331E+01  5.93E-01    5.69   44.06   0.07    1.08   3
 !-----------------------------------------------------------------------------

and W2:

| 200000 calls, 366 channels, 14 dimensions, 20 bins, stratified = T
|=============================================================================|
| It      Calls  Integral[fb]  Error[fb]   Err[%]    Acc  Eff[%]   Chi2 N[It] |
|=============================================================================|
   1     200000  1.0758844E+01  4.98E-01    4.63   20.70*   0.42
|-----------------------------------------------------------------------------|
   1     200000  1.0758844E+01  4.98E-01    4.63   20.70    0.42
|-----------------------------------------------------------------------------|
   2     200000  1.1499868E+01  5.37E-01    4.67   20.90    0.43
   3     200000  1.0853466E+01  2.44E-01    2.25   10.05*   0.34
   4     200000  1.1619145E+01  5.22E-01    4.49   20.08    0.58
   5     200000  1.0819072E+01  1.83E-01    1.69    7.54*   0.80
   6     200000  1.1046300E+01  2.37E-01    2.14    9.58    0.78
   7     200000  1.1488141E+01  2.39E-01    2.08    9.32*   0.75
   8     200000  1.1085825E+01  1.73E-01    1.56    6.98*   0.95
   9     200000  1.1162951E+01  1.57E-01    1.41    6.29*   1.14
  10     200000  1.1495043E+01  1.71E-01    1.49    6.67    1.03
  11     200000  1.1352598E+01  1.34E-01    1.18    5.29*   1.17
|-----------------------------------------------------------------------------|
  11    2000000  1.1203114E+01  6.29E-02    0.56    7.94    1.17    1.55  10
|-----------------------------------------------------------------------------|
  12     200000  1.1202662E+01  1.32E-01    1.18    5.28*   1.17
  13     200000  1.1075765E+01  1.20E-01    1.08    4.85*   1.37
  14     200000  1.1191362E+01  1.25E-01    1.12    5.00    1.36
|=============================================================================|
  14     600000  1.1152666E+01  7.25E-02    0.65    5.03    1.36    0.32   3
|=============================================================================|
| WHIZARD run finished.
|=============================================================================|

Apparently, W1 lacks important integration channels.

Now it's time to close this ticket.

[Juergen Reuter]

Milestone v2.0-rc1 deleted