Mismatch between event weights and total cross section

[cweiss]

In the default event normalization mode sigma, events are normalized according to

<sum_weights> = N x σ,

therefore, σ = <sum_weights> / N. Summing up weights during event generation using the option

?check_event_weights_against_xsection = true

reveals that this property is fulfilled for processes without structure functions, but finds discrepancies for processes with structure functions, e.g. ISR or parton densities.

Attached are Sindarins and whizard logs specifying this issue.

[kilian]

Reproduced, any simple test case.

[kilian]

OK, I now have a suspicion about what's going on.

The problem is not caused by structure functions being active. The problem appears for any process when cuts are present.

I suspect that the cause is the rejection loop around the weighted event generation call. The rejection trashes all invalid events, which should include events that don't pass the cuts. But they likely count towards the total event number. In effect, summing up only nonzero weights overcounts the total weight.

I'll investigate this further and try to come up with a solution. If my suspicion turns out to be true, only the overall normalization is wrong, no problem for distributions etc.

[kilian]

Confirmed, removing the REJECTION loop in process_mci_entry_generate_weighted_event fixes the cross section value.

That's obviously a hack, not the final solution. I'll have to think about it.

[cweiss]

Unfortunately, I cannot confirm this. For the process uu -> zz, which converges nicely without cuts, I get the following:

Integrated Cross Section: 9.0732779E+02 + -5.68E+0 Reconstructed Cross Section: 952.19927529264567 +- 2.8668267152136688

See the Sindarin in the attachement.

[kilian]

No, that's also covered by the quick fix. My results for uuzz.sin (replacing LHAPDF by pdf_builtin, since I don't have LHAPDF installed):

• integrated: 8.1791110E+02 +- 4.85E+00
• reconstructed: 8.1826371510520937E+02 +- 2.0950741926522474

Point is, that PDF structure functions always imply cuts, because events outside of the PDF allowed range are marked as invalid.

My problem is that (i) we don't really want events with zero weight and invalid kinematics in the event file, but (ii) the fraction of those is known only after the run, while the weights have to be written event by event.

[Juergen Reuter]

Wasn't there a similar problem for the rescanning/reweighting procedures? That the total weight for the reweighted sample was known only after the full sample was rescanned but you had to know the new weight for every event already? couldn't you adapt the solution from there?

[kilian]

Sort of.

I guess that if all invalid events are written out or just counted as events, there is no problem. Look at this diagnostics:

| Events: generating 100000 weighted, unpolarized events ...
| Events: event normalization mode 'sigma'
|         ... event sample complete.
| Events: closing raw file 'uuee.evx'
 Reconstructed total cross section with  105243  events: 
   8.1826371510520937E+02  +-    2.0950741926522474

The number of actual events is higher than requested, then the results fit. So we could drop the events at a later stage in the program flow. We have to drop them at some point because they are invalid. Nevertheless, the actual number has to be known somewhere, and it might be necessary to communicate it together with the event file, or alternatively all weights have to be renormalized.

[cweiss]

Okay, I get the problem now. The question is, whether this is actually a bug. For our applications, we would use the integrated cross section to scale all histograms. However, we might think of applications where the user wants to reconstruct the total cross section on-the-fly via the events.

I think the best solution is really to write out zero-weighted events, which can be switched on in Sindarin. The other way would be to keep track of the vetoed events, but the output formats do not support to deal with just a single integer like that. If you are OK with that, I would start with the implementation.

[kilian]

Maybe yes.

The events must not be used in actual calculations, the kinematics and other parameters may be invalid. I didn't check whether there can be a problem with computations inside Whizard; in any case for external event handlers you can't be sure whether zero weight will be caught. So the default should be off as before.

So for now, implement a switch that will write those events, if only for being counted correctly. Keep the "check weight" switch, that is a useful addition anyway (I might make it a bit more fitting with the rest). We can find a good solution in the upcoming weeks.

I don't have time right now, so go ahead.