Event output totally screwed up

[kilian]

When photons are radiated (ISR), the event record appears incorrect in various output formats. Photons appear twice, and relations are broken. Reported for Stdhep and HepMC. (J.J. Blaising)

[kilian]

This is actually a problem with the combination ISR + intrinsic PYTHIA 6 + event output. The event records are corrupted.

[Juergen Reuter]

Can this be due to the fact that this handled via intermediate LHE scratch files that this gets corrupted on the way WHIZARD->PYTHIA or the other way round?

[kilian]

Replying to jr_reuter:

Can this be due to the fact that this handled via intermediate LHE scratch files that this gets corrupted on the way WHIZARD->PYTHIA or the other way round?

Possibly. There are too many intermediate formats involved. One has to print the event after each transformation step and compare.

[Juergen Reuter]

My new suspicion is the function particle_set_reduce used just before the shower. This subroutine does a lot, namely copying around particles, looking for the status etc.

[Bijan Chokoufe Nejad]

OK. So far I can say the following:

• particle_set_reduce does not change the particle_set for the given sindarin file when I activate ?ps_fsr_active = true and ?ps_use_PYTHIA_shower = true.

• The "copied photons" appear after apply_PYTHIAshower_particle_set. But in the particle set, the original photons are now marked as virtual just as the original final state particles, as seen below:

...
 Particle 5 [v] f(22)
 E =   2.747151184467E-04
 P =   0.000000000000E+00  0.000000000000E+00  2.747151184467E-04
 T =   0.000000000000E+00
 Parents:  1
 Children: 9
 Particle 6 [v] f(22)
 E =   1.465052415406E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.465052415406E+03
 T =   0.000000000000E+00
 Parents:  2
 Children: 10
 Particle 7 [v] f(13)
 E =   4.353680841389E+02
 P =  -7.044377250829E+01  1.915309377830E+02  3.845763023248E+02
 T =   1.116369516608E-02
 Parents:  3 4
 Children: 11
 Particle 8 [v] f(-13)
 E =   1.099579225740E+03
 P =   7.044377250829E+01 -1.915309377830E+02  1.080475838370E+03
 T =   1.116369516608E-02
 Parents:  3 4
 Children: 12
 Particle 9 [o] f(22)
 E =   2.747151184500E-04
 P =   0.000000000000E+00  0.000000000000E+00  2.747151184500E-04
 T =   0.000000000000E+00
 Parents:  5
 Particle 10 [o] f(22)
 E =   1.465052415400E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.465052415400E+03
 T =   0.000000000000E+00
 Parents:  6
...

• Is this not the desired behavior? Shouldn't virtual particles be removed before the mokka file is written? I don't know what the policy is for this.

• I think, the shower does basically what SS has written in front of shower_add_lhef_to_particle_set

  @ Read in the LHE file opened in unit [[u]] and add the final
  particles to the [[particle_set]], the outgoing particles of the existing
  [[particle_set]] are set to be virtual before the outgoing particles from the 
  LHEF are added to the [[particle_set]] as outgoing particles. At the same 
  time, it is searched for pairs of existing and new particles with
  equal momenta and PDG code, these pairs are marked as connected. All 
  remaining existing former outgoing particles are set to be the mothers of all 
  remaining new outgoing particles.
  

So please clarify what you mean by corrupted and what we should change.

[Juergen Reuter]

This is the comment from Jean-Jacques Blaising and Philip Roloff:

1.) The ISR photons appear twice with generator status 1. In the 
    example: index = 9,10 and index = 12,13. Using WHIZARD 1.95, 
    the ISR photons appeared only once with generator status 1 
    as expected.

2.) When generating events for the same process using WHIZARD 1.95, we 
    see the muons also with generator status 2 which is not the case 
    for WHIZARD 2.2.3. This information is needed to identify the 
    FSR photons.

3.) There seems to be a problem with the mother-daughter relations 
    of the photons.

[Bijan Chokoufe Nejad]

OK at first the revocation:

• There is not one but two copies of the ISR photons. I didn't spot the second photon that is also marked as outgoing by the shower. The virtual one is fine if I understand WK correctly.

In fact, the copy is already in Pythias event record although the .lhe that is fed into Pythia does not have these copies. However, some things are wrong with it that might cause the issue:

• The ISR photons have no mother particles. This information is lost as the shower calls hepeup_from_particle_set to produce the .lhe which calls particle_set_to_hepevt_form. The photons in the particle set lose in particle_set_to_hepevt_form their parents.

• Potential reason for this loss is the fact that the initial electron beams (parents of the ISR) are removed from the set. Using keep_beams = .true. does not change this behavior at all.

• @WK: Can you have a look at hepeup_from_particle_set given the isr_1.sin and activating Pythia FSR? There you can see the original particle set and how the photons lose their parents. I don't get the logic inside particle_set_to_hepevt_form.

Related to this is also point 2.) of the user report. They expect to see the muons with status 2 AND status 1, i.e. before and after emission of the FSR photons by Pythia. This information is in the particle set but the mokka file format uses hepevt_from_event -> hepevt_from_particle_set -> particle_set_to_hepevt_form where also the virtual particles are dropped.

[Bijan Chokoufe Nejad]

Addendum: The ISR photons lose their mothers even without the shower and with lhe as output. This is expected since in eio_lhef_output we call hepeup_from_event -> hepeup_from_particle_set -> particle_set_to_hepevt_form.

[kilian]

Replying to bchokoufe:

OK at first the revocation:

• There is not one but two copies of the ISR photons. I didn't spot the second photon that is also marked as outgoing by the shower. The virtual one is fine if I understand WK correctly.

In fact, the copy is already in Pythias event record although the .lhe that is fed into Pythia does not have these copies. However, some things are wrong with it that might cause the issue:

• The ISR photons have no mother particles. This information is lost as the shower calls hepeup_from_particle_set to produce the .lhe which calls particle_set_to_hepevt_form. The photons in the particle set lose in particle_set_to_hepevt_form their parents.

What _should_ be the correct assignment for those ISR photons? Should they get the incoming partons as their parents? Or should they become orphan outgoing particles?

• Potential reason for this loss is the fact that the initial electron beams (parents of the ISR) are removed from the set. Using keep_beams = .true. does not change this behavior at all.

Well, the latter is probably an error.

• @WK: Can you have a look at hepeup_from_particle_set given the isr_1.sin and activating Pythia FSR? There you can see the original particle set and how the photons lose their parents. I don't get the logic inside particle_set_to_hepevt_form.

Yes, I'll try to clean up this mess.

Related to this is also point 2.) of the user report. They expect to see the muons with status 2 AND status 1, i.e. before and after emission of the FSR photons by Pythia. This information is in the particle set but the mokka file format uses hepevt_from_event -> hepevt_from_particle_set -> particle_set_to_hepevt_form where also the virtual particles are dropped.

[Bijan Chokoufe Nejad]

I would say that the initial-state-radiated photons should get the incoming partons as parents. My guess is that Pythia doesn't know what to do with orphans. Maybe you remember how the ISR photons were written out with W1.

[kilian]

As JR already mentioned, particle_set_reduce drops the beams and thus leaves the radiated photons as orphans. The shower code doesn't set the keep_beams option which probably would have preserved the relations. I.e., the particle set is given to PYTHIA without beams and including orphans.

Note that particle_set_to_hepevt_form doesn't modify the record at all, in this case.

Why does this work with hadrons? There is also code in shower_add_lhef_to_particle_set ...

[Bijan Chokoufe Nejad]

?

I added the keep_beams option for testing purposes. I have not checked this in.

I cannot reproduce what you are reporting. With r6554 and the very sindarin File that is attached in this ticket, I see that the ISR photons have no parents in the mokka file

 1 22 0 0  0.0000000000E+00  0.0000000000E+00  3.3986659634E+02  0.0000000000E+00
 1 22 0 0  0.0000000000E+00  0.0000000000E+00 -1.5494924427E-04  0.0000000000E+00

or alternatively if I activate LHE

 22 1 0 0 0 0  0.0000000000E+00  0.0000000000E+00  3.3986659634E+02  3.3986659634E+02  0.0000000000E+00  0.0000000000E+00  9.0000000000E+00
 22 1 0 0 0 0  0.0000000000E+00  0.0000000000E+00 -1.5494924427E-04  1.5494924427E-04  0.0000000000E+00  0.0000000000E+00  9.0000000000E+00

And for this the shower and therefore also particle_set_reduce is not called.

Have you actually used particle_set_write to verify that the beams are dropped? Because I have looked at it and particle_set_reduce did not change the particle set. It was only changed after hepeup_from_particle_set which is consistent with the case in the given sindarin.

[kilian]

No, I did exactly that.

This is generated by WHIZARD, with beams and correct relations.

 particle set in applya_pythiashower_particle_set
 Particle set:
------------------------------------------------------------------------
 Particle 1 [b] f(11)
 E =   1.500000000000E+03
 P =   0.000000000000E+00  0.000000000000E+00  1.500000000000E+03
 T =   2.611179340090E-07
 Children: 3 5
 Particle 2 [b] f(-11)
 E =   1.500000000000E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.500000000000E+03
 T =   2.611179340090E-07
 Children: 4 6
 Particle 3 [i] f(11)
 E =   1.373806765518E+03
 P =   0.000000000000E+00  0.000000000000E+00  1.373806765518E+03
 T =   2.611179340090E-07
 Parents:  1
 Children: 7 8
 Particle 4 [i] f(-11)
 E =   1.499564621721E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.499564621721E+03
 T =   2.611179340090E-07
 Parents:  2
 Children: 7 8
 Particle 5 [o] f(22)
 E =   1.261932344823E+02
 P =   0.000000000000E+00  0.000000000000E+00  1.261932344823E+02
 T =   0.000000000000E+00
 Parents:  1
 Particle 6 [o] f(22)
 E =   4.353782790145E-01
 P =   0.000000000000E+00  0.000000000000E+00 -4.353782790145E-01
 T =   0.000000000000E+00
 Parents:  2
 Particle 7 [o] f(13)
 E =   1.423871547539E+03
 P =  -7.915255655706E+02  1.161051863403E+03  2.299043763453E+02
 T =   1.116369516608E-02
 Parents:  3 4
 Particle 8 [o] f(-13)
 E =   1.449499839700E+03
 P =   7.915255655706E+02 -1.161051863403E+03 -3.556622325486E+02
 T =   1.116369516608E-02
 Parents:  3 4

The shower routine first calls particle_set_reduce w/o the beam option, producing

 particle set in hepeup_from_particle_set
 Particle set:
------------------------------------------------------------------------
 Particle 1 [i] f(11)
 E =   1.373806765518E+03
 P =   0.000000000000E+00  0.000000000000E+00  1.373806765518E+03
 T =   2.611179340090E-07
 Children: 5 6
 Particle 2 [i] f(-11)
 E =   1.499564621721E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.499564621721E+03
 T =   2.611179340090E-07
 Children: 5 6
 Particle 3 [o] f(22)
 E =   1.261932344823E+02
 P =   0.000000000000E+00  0.000000000000E+00  1.261932344823E+02
 T =   0.000000000000E+00
 Particle 4 [o] f(22)
 E =   4.353782790145E-01
 P =   0.000000000000E+00  0.000000000000E+00 -4.353782790145E-01
 T =   0.000000000000E+00
 Particle 5 [o] f(13)
 E =   1.423871547539E+03
 P =  -7.915255655706E+02  1.161051863403E+03  2.299043763453E+02
 T =   1.116369516608E-02
 Parents:  1 2
 Particle 6 [o] f(-13)
 E =   1.449499839700E+03
 P =   7.915255655706E+02 -1.161051863403E+03 -3.556622325486E+02
 T =   1.116369516608E-02
 Parents:  1 2

The photons are orphaned since now the beams are missing.

This is not changed by

 particle set after particle_set_to_hepevt_form
 Particle set:
------------------------------------------------------------------------
 Particle 1 [i] f(11)
 E =   1.373806765518E+03
 P =   0.000000000000E+00  0.000000000000E+00  1.373806765518E+03
 T =   2.611179340090E-07
 Children: 5 6
 Particle 2 [i] f(-11)
 E =   1.499564621721E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.499564621721E+03
 T =   2.611179340090E-07
 Children: 5 6
 Particle 3 [o] f(22)
 E =   1.261932344823E+02
 P =   0.000000000000E+00  0.000000000000E+00  1.261932344823E+02
 T =   0.000000000000E+00
 Particle 4 [o] f(22)
 E =   4.353782790145E-01
 P =   0.000000000000E+00  0.000000000000E+00 -4.353782790145E-01
 T =   0.000000000000E+00
 Particle 5 [o] f(13)
 E =   1.423871547539E+03
 P =  -7.915255655706E+02  1.161051863403E+03  2.299043763453E+02
 T =   1.116369516608E-02
 Parents:  1 2
 Particle 6 [o] f(-13)
 E =   1.449499839700E+03
 P =   7.915255655706E+02 -1.161051863403E+03 -3.556622325486E+02
 T =   1.116369516608E-02
 Parents:  1 2

and appears in the HEPEUP common block.

My main question is: how is this done for hadronic collisions? I don't have a working LHAPDF here, so can't check with the PYTHIA shower. In hadronic showered events, do the beam remnants appear, and with which parents?

[Juergen Reuter]

Why don't you just use the intrinsic PDFs?

[kilian]

Replying to jr_reuter:

Why don't you just use the intrinsic PDFs?

******************************************************************************
******************************************************************************
*** FATAL ERROR: Builtin PDFs cannot be used for PYTHIA showers,
     aborting simulation
******************************************************************************
******************************************************************************

[Bijan Chokoufe Nejad]

And what do you get without the shower? Can you reproduce the mokka and the LHE or are they different?!

[Bijan Chokoufe Nejad]

I have in the shower source

      call particle_set_reduce (particle_set, pset_reduced)
      if (debug) then
         print *, "apply_PYTHIAshower_particle_set: after particle_set_reduce"
         call particle_set_write (particle_set)
      end if

and get

 apply_PYTHIAshower_particle_set: after particle_set_reduce
 Particle set:
------------------------------------------------------------------------
 Particle 1 [b] f(11)
 E =   1.500000000000E+03
 P =   0.000000000000E+00  0.000000000000E+00  1.500000000000E+03
 T =   2.611179340090E-07
 Children: 3 5
 Particle 2 [b] f(-11)
 E =   1.500000000000E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.500000000000E+03
 T =   2.611179340090E-07
 Children: 4 6
 Particle 3 [i] f(11)
 E =   3.103291353116E+02
 P =   0.000000000000E+00  0.000000000000E+00  3.103291353112E+02
 T =   2.611179340090E-07
 Parents:  1
 Children: 7 8
 Particle 4 [i] f(-11)
 E =   1.499999382955E+03
 P =   0.000000000000E+00  0.000000000000E+00 -1.499999382955E+03
 T =   2.611179340090E-07
 Parents:  2
 Children: 7 8
 Particle 5 [o] f(22)
 E =   1.189670864688E+03
 P =   0.000000000000E+00  0.000000000000E+00  1.189670864688E+03
 T =   0.000000000000E+00
 Parents:  1
 Particle 6 [o] f(22)
 E =   6.170451878673E-04
 P =   0.000000000000E+00  0.000000000000E+00 -6.170451878673E-04
 T =   0.000000000000E+00
 Parents:  2
 Particle 7 [o] f(13)
 E =   6.829284777962E+02
 P =   5.939081559318E+02 -2.186121864556E+02 -2.566575712348E+02
 T =   1.116369516608E-02
 Parents:  3 4
 Particle 8 [o] f(-13)
 E =   1.127400040470E+03
 P =  -5.939081559318E+02  2.186121864556E+02 -9.330126764087E+02
 T =   1.116369516608E-02
 Parents:  3 4

So immediately after particle_set_reduce, I get the full particle set that is put in !?

EDIT: Now I see. I output particle_set instead of pset_reduced *GRMPF*

[Bijan Chokoufe Nejad]

To answer your question with hadronic events. For DY, we have at first

 Particle set:
------------------------------------------------------------------------
 Particle 1 [b] f(2212)
 E =   9.800000000000E+02
 P =   0.000000000000E+00  0.000000000000E+00  9.800000000000E+02
 T =   0.000000000000E+00
 Children: 3 5
 Particle 2 [b] f(-2212)
 E =   9.800000000000E+02
 P =   0.000000000000E+00  0.000000000000E+00 -9.800000000000E+02
 T =   0.000000000000E+00
 Children: 4 6
 Particle 3 [i] f(2)c(1 )
 E =   2.709866398502E+01
 P =   0.000000000000E+00  0.000000000000E+00  2.709866398502E+01
 T =   0.000000000000E+00
 Parents:  1
 Children: 7 8
 Particle 4 [i] f(-2)c(-1 )
 E =   9.193815484270E+01
 P =   0.000000000000E+00  0.000000000000E+00 -9.193815484270E+01
 T =   0.000000000000E+00
 Parents:  2
 Children: 7 8
 Particle 5 [x] f(-92)c(-1 )
 E =   9.529013360150E+02
 P =   0.000000000000E+00  0.000000000000E+00  9.529013360150E+02
 T =   0.000000000000E+00
 Parents:  1
 Particle 6 [x] f(92)c(1 )
 E =   8.880618451573E+02
 P =   0.000000000000E+00  0.000000000000E+00 -8.880618451573E+02
 T =   0.000000000000E+00
 Parents:  2
 Particle 7 [o] f(11)
 E =   4.443339138071E+01
 P =  -2.584589284574E+00  4.410572094326E+01 -4.725626692353E+00
 T =   2.611179340090E-07
 Parents:  3 4
 Particle 8 [o] f(-11)
 E =   7.460342744702E+01
 P =   2.584589284574E+00 -4.410572094326E+01 -6.011386416533E+01
 T =   2.611179340090E-07
 Parents:  3 4

and after partice_set_reduce it is

 Particle 1 [i] f(2)c(1 )
 E =   2.709866398502E+01
 P =   0.000000000000E+00  0.000000000000E+00  2.709866398502E+01
 T =   0.000000000000E+00
 Children: 3 4
 Particle 2 [i] f(-2)c(-1 )
 E =   9.193815484270E+01
 P =   0.000000000000E+00  0.000000000000E+00 -9.193815484270E+01
 T =   0.000000000000E+00
 Children: 3 4
 Particle 3 [o] f(11)
 E =   4.443339138071E+01
 P =  -2.584589284574E+00  4.410572094326E+01 -4.725626692353E+00
 T =   2.611179340090E-07
 Parents:  1 2
 Particle 4 [o] f(-11)
 E =   7.460342744702E+01
 P =   2.584589284574E+00 -4.410572094326E+01 -6.011386416533E+01
 T =   2.611179340090E-07
 Parents:  1 2

So the initial protons and the beam remnants are removed and the hard process remains. I will have a closer look at the reduce now. Though I still not understand how the photons are orphaned in the case without the shower.

[kilian]

OK. The obvious difference is that the ISR photons have status outgoing, while the proton beam remnants have the status beam remnant. The latter are removed, outgoing particles are kept.

But there must be some place where beam remnants appear again (as virtual objects with their real decay products), right? What are their parents after all shower/hadronization? Once we know this, the ISR photon should be removed and re-inserted analogously.

[kilian]

A more radical change: let the ISR (and CIRCE, etc.) structure function radiate particles with a new flavor code ISR_PHOTON, etc., analogous to HADRON_REMNANT. We could keep this inside WHIZARD and transform into ordinary photon etc. for event analysis and for output formats.

Currently, the identification of the beam remnant is done based on the flavor code. So this would solve part of the problem.

The additional benefit would be that the ISR_PHOTON pseudoparticle could actually fragment into several photons according to the correct distribution, if someone writes that code.

Opinions?

[Bijan Chokoufe Nejad]

Yes the beam remnants are reinserted as virtual particles and keep their old parents, i.e. the initial protons. They are definitly there after apply_PYTHIAshower_particle_set. I think using ISR_PHOTON analoguous to HADRON_REMNANT is fine.

[Bijan Chokoufe Nejad]

So in shower_add_lhef_to_particle_set the remnants are made virtual:

       temp_prt (i) = particle_set_get_particle (particle_set, i)
       if (particle_get_status (temp_prt (i)) == PRT_OUTGOING .or. &
            particle_get_status (temp_prt (i)) == PRT_BEAM_REMNANT) then
          call particle_reset_status (temp_prt (i), PRT_VIRTUAL)
          available_parents (i) = i
       end if

The original particle_set is used here, not the one which has been stripped down to feed it into Pythia and the event information of the LHE is added to it. So I guess we just have to avoid that the ISR_PHOTON is put into Pythia by extending particle_set_reduce or give it another status code? Of course we have to put in Pythia the electrons _after_ emission of the ISR_PHOTON.

[kilian]

My current solution tags the flavor quantum number of the appropriate particles as radiated. In the particle sets, those flags translate to status beam_remnant. I'll yet have to code how this status is processed further. The idea is that unresolved beam remnants are translated to outgoing, but there is left room for an event transform that digests those.

[kilian]

Several issues are fixed by r6559.

• HepMC events now support the signal_process_vertex flag. WHIZARD sets this flag and requires it on input.
• Beam remnants (hadronic, ISR photons, EPA electrons, etc.) are consistently tagged as such. Upon event output, they are either re-tagged as outgoing (real) or as virtual particles.
• This choice is controlled by a new ?keep_remnants flag.
• Event input routines are able to reconstruct this structure, in particular if the event has been written by WHIZARD.
• Some dead code eliminated.

I'll check what remains to be done for the original problem.

[kilian]

Current status, looking at the original ISR + PYTHIA shower: the code in apply_PYTHIAshower_particle_set does screw up the mother-daughter relations for photons. The resulting particle set is not consistent with PYTHIA's internal particle record from which it should originate. So, the problem is not yet solved.

[kilian]

I'm afraid the damage is done in PYLHEO, which is apparently a modified adaptation of PYTHIA's routine PYLHEF. In an attempt to write valid LHEF output, the program drops the intermediate relations (i.e., whether the photons are radiated from muons or from electrons) and it reassigns one of the ISR photons to the final state (dropping the other one which has only a tiny amount of energy). Merging the result into the original particle record then produces garbage in shower_add_lhef_to_particle_set.

I'm lost. Can anyone debug this code? It is also used after PYTHIA hadronization, obviously, and apparently does work with hadronic events. A possible but ugly workaround is to forbid PYTHIA hadronization in connection with ISR.

[Bijan Chokoufe Nejad]

OK, I am a bit confused now. You said that ISR photons are now treated as remnants. The remnants shouldn't enter Pythia at all but be eliminated by particle_set_reduce. Right?

So the missing step would eventually be to make the ISR photon remnants in shower_add_lhef_to_particle_set outgoing instead of virtual if this is desired?

Are the mother daughter relations of the ISR photons without shower and stdhep/mokka/lhef output already fixed?

[Juergen Reuter]

Replying to kilian:

Can you give more information. What is the input file. How should the output look like? At which intermediate steps are you looking

[kilian]

Replying to jr_reuter:

Replying to kilian:

Can you give more information. What is the input file.

Done, replaced the previous isr_1.sin.

How should the output look like?

(Use any event format for output)

The output should contain two ISR photons (one of those with tiny momentum) attached to the incoming beams, photons attached to the incoming electrons of the hard process, and photons attached to intermediate muons, which should then resolve into real muons.

Currently there is only one pair of muons, one of the ISR photons is lost, and the other ISR photon is misconnected; there are actually two copies of unrelated photons with status code 1, as in the user's report.

At which intermediate steps are you looking

Check the status before and after the call to PYLHEO. The PYTHIA internal state is printed by pylist(2) and looks ok. After PYLHEO, rewind and print the generated scratch file line by line to screen for checking, or open a named file instead of the scratch file before this. Note that the following routine LHEF -> particle_set screws up the event further, but this should not be a surprise.

[kilian]

Replying to bchokoufe:

OK, I am a bit confused now. You said that ISR photons are now treated as remnants. The remnants shouldn't enter Pythia at all but be eliminated by particle_set_reduce. Right?

Yes. If PYTHIA doesn't receive beams, it shouldn't receive radiated particles either. I think that's what happens with hadron remnants.

So the missing step would eventually be to make the ISR photon remnants in shower_add_lhef_to_particle_set outgoing instead of virtual if this is desired?

PYTHIA (I think!) also generates beam entries and attaches ISR photons, so don't double count.

Are the mother daughter relations of the ISR photons without shower and stdhep/mokka/lhef output already fixed?

Without shower, everything looks fine so far. ISR photons can only be written if beam particles are also present, otherwise they wouldn't have parents.

[Juergen Reuter]

Replying to kilian:

Ok, this is what I see

 13 7 0  1.0000000000E+00
 2 11 0 0 9 13 0
  0.0000000000E+00  0.0000000000E+00  1.5000000000E+03  1.5000000000E+03  5.1099700000E-04
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 2 -11 0 0 9 13 0
  0.0000000000E+00  0.0000000000E+00 -4.2590677412E+02  4.2590677412E+02  5.1099700000E-04
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 2 22 0 0 7 7 0
  0.0000000000E+00  0.0000000000E+00  2.1826810943E-11  2.1826810943E-11  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 2 22 0 0 8 8 0
  0.0000000000E+00  0.0000000000E+00 -1.0740932259E+03  1.0740932259E+03  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 2 22 0 0 7 7 0
 -4.2389809877E-04 -6.8460976335E-03  1.6282983612E-02  1.7668737864E-02 -2.3283064365E-10
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 2 22 0 0 8 8 0
 -0.0000000000E+00 -0.0000000000E+00 -1.0740932259E+03  1.0740932259E+03  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 22 0 0 0 0 0
 -4.2389809877E-04 -6.8460976335E-03  1.6282983612E-02  1.7668737864E-02  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 22 0 0 0 0 0
 -0.0000000000E+00 -0.0000000000E+00 -1.0740932259E+03  1.0740932259E+03  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 22 1 2 0 0 0
  1.7311056524E-03  1.3807562163E-03 -7.2042800982E-04  2.3285684499E-03  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 13 1 2 0 0 0
 -4.9628490194E+02 -2.7364901119E+02  1.2149075458E+03  1.3405904823E+03  1.0565838900E-01
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 22 1 2 0 0 0
 -3.1257233032E-01 -1.4311355498E-01  7.4174794887E-01  8.1754080690E-01  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 -13 1 2 0 0 0
  4.9659606449E+02  2.7379755461E+02 -1.4157159304E+02  5.8447863887E+02  1.0565838900E-01
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00
 1 22 1 2 0 0 0
  1.0257478853E-04  3.5475084591E-05 -3.7423588847E-05  1.1480676753E-04  0.0000000000E+00
  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00  0.0000000000E+00

[Juergen Reuter]

Now I need some help to understand the entries: The first line #part. (13), outgoing particles (7), beam remnants (0, this is always set like this, I remember from the code). and the weight (1.00000). Then the event: The first entry is the status code, where 1 is outgoing and 2 is resonant. So there are no resonant muons, but only resonant electrons and resonant photons (which are actually what? I guess the two ISR photons!?) Why does one of them have so tiny momentum?

[Juergen Reuter]

In WHIZARD 1 the PYEVTC routine of PYTHIA was used, which according to the PYTHIA code transfers the contents of the PYTHIA event record to the standard event common blocks. Maybe we should have a look there, there is some code concerning 'filling in missing mother/daughter' information, and also status codes being written out.

[kilian]

Regarding the above event: particles 3 and 4 are virtual photons with daughters 7 and 8, but 5 and 6 are also virtual photons with daughters 7 and 8. This is wrong: photons 3 and 4 are actually the ISR photons, but photon 5 = 7 is FSR, photon 4 = 6 = 8 is again ISR. Also the intermediate muons are missing.

[Juergen Reuter]

I found the culprit: it is SS's modifications from the PYLHEF to the PYLHE) routine. (1) kicking out the ultrasoft (zero-energy) photons from the PYTHIA QED shower, and adding the color information (which I'm not sure is really necessary). Trying out some things now. I guess, we can throw out those photons later in the particle set, guaranteeing that the mother-daughter information is not screwed up. One remark: can't we do the photon shower by ourselves??

[Bijan Chokoufe Nejad]

The photon QED shower should be as far as I can see straightforward. Different alpha and splitting function and no color connection. But otherwise same architecture.

[Juergen Reuter]

Next piece of information: the mother-daughter information in PYLHEO might be screwed up (and I now understand why actually, cf. above). But the MOTHUP from the LHEF scratch file that is written is NOT used at all in shower_add_lhef_to_particle_set. This information needs to be taken into account!

[kilian]

Replying to bchokoufe:

The photon QED shower should be as far as I can see straightforward. Different alpha and splitting function and no color connection. But otherwise same architecture.

Yes. When we address the photon shower, we would eventually go beyond PYTHIA and do what KK does, so it is a separate project.

And no: in any case, using PYTHIA for shower should yield meaningful results.

[Juergen Reuter]

There is a solution that already works for uncolored final states as well as colored final states without hadronization. For hadronziation, it seems that hadronic decays are posing a problem at the moment. Have to investigate a little further, but pretty sure to close this very soon.

[Juergen Reuter]

Resolved in r6591. Now, double entries are consistently removed and the moghter-daughter relations from PYTHIA's output are (hopefully) correctly transferred to the WHIZARD particle set. Hope is nevertheless that this is smoother/eleganter/faster (?) with PYTHIA8. Closing this one.

[Juergen Reuter]

Apparently, this was to fast. Seems that problems are sadly still with us.

[Juergen Reuter]

One first observation: the ISR photons are not recognized as being present. They are not passed to PYTHIA6 due to WK's catch mechanism, but PYTHIA6 reconstructs them from the whole setup of the process/event. But then the precision inside PYTHIA6 is not as high as in WHIZARD, and hence there is a difference at the level of 1E-3 between the momenta of the ISR photons from PYTHIA6 and those from WHIZARD. Hence, they appear twice.

[Juergen Reuter]

There are so many weirdnesses. In the LHE format, ?keep_remnants doesn't make a difference, but ?keep_beams adds another six particles to the output. Apparently not only particles with LHE status -9, but also with status 3.

[Juergen Reuter]

Clearly, the function particle_set_to_hepevt_form is utterly broken. Correct mother-daughter relations come out completely screwed. WK, for LHA and LHEF, and the formats using HEPRUP/HEPEVT common blocks, does one really have to bring children into contiguous order?

[kilian]

Replying to jr_reuter:

One first observation: the ISR photons are not recognized as being present. They are not passed to PYTHIA6 due to WK's catch mechanism, but PYTHIA6 reconstructs them from the whole setup of the process/event. But then the precision inside PYTHIA6 is not as high as in WHIZARD, and hence there is a difference at the level of 1E-3 between the momenta of the ISR photons from PYTHIA6 and those from WHIZARD. Hence, they appear twice.

Yes, that's reasonable. PYTHIA does essentially the same thing that WHIZARD does, it reconstructs an ISR shower explicitly (which here consists of a single photon). One has to choose whether to use the PYHTIA or the WHIZARD ISR photons. The WHIZARD ISR photons are now tagged as such.

Regarding ?keep_remnants: this switch categorizes the remnants as outgoing. Probably, it makes a real difference only for hadron beams. BTW, LHA/LHEF were obviously not intended for e+e-.

Regarding contiguity: if the event format only stores FIRST_DAUGHTER, LAST_DAUGHTER or similar, then yes, they better be in contiguous order. That's the reason why HepMC (and LCIO?) have chosen to use a node-tree model.

[Juergen Reuter]

With r6738, daughters and parents seem to appear correctly in the HEPRUP/HEPEUP as well as in the HEPEVT-based event formats. What is still missing, is that the status codes of the particles need to be adjusted correctly, particularly in the routine lhef_file_to_particle_set. An abellimento would be to rewrite transferral routines without the detour of a LHE scratch file.

[Juergen Reuter]

Mother-daughter relations for the WHIZARD shower seem completely screwed up. There all final state partons have the first initial state particle as mother (cf. the unit test output for shower_2). I believe that this is a problem of the transfer routines from the WHIZARD shower back to the particle set.

[Juergen Reuter]

Some more information: the routine transferring the output of the WHIZARD shower back into the particle_set gives all final state particles the incoming particles as mothers and daughters. This is, I guess, the final verdict to replace this by a proper routine, transferring the shower state back into the particle_set.

[Juergen Reuter]

BACN, what is the status here. Something still missing?

[Juergen Reuter]

All known issues (also the latest ones concerning stable vs. resonant tau leptons by Jean-Jacques Blaising and Philipp Roloff) have been resolved. Closing for now.