Compilation fails for large LHC process (code size, address space problems)

[Juergen Reuter]

The following SINDARIN file first of all needs (on a Xenom QuadCore?) 38 mins to produce the matrix element code, as it has 400 subprocesses,90 per cent of which are forbidden or so. Secondly, it produces invalid code (probably a line continuation problem!?) which does not compile. Here is the SINDARIN file:

model = SM

alias q  = u:d:s:c
alias Q  = U:D:S:C
alias lep = e1:E1
alias neu = n1:N1

process fail = g, g => lep,neu,b,B,q,Q,q,Q

compile

These are the error messages:

| Compiling process library 'processes'                                                                           
libtool: compile:  gfortran -c -I/home/reuter/local/lib/whizard/mod/models -I/home/reuter/local/include/omega -I/home/reuter/local/lib/whizard/mod/whizard-core -I/home/reuter/local/lib/whizard/mod/misc -g -O2 fail.f90  -fPIC -o .libs/fail.o                                                                                                      
fail.f90:77747.10:                                                                                                

         + v_ff((-gs),owf_u2b__4_2349,owf_u2_3__5) &
          1                                         
Fehler: Ungltiges Zeichen in Name bei (1)          
fail.f90:77749.132:                                 
.....
.....
.....
Schwerwiegender Fehler: Fehlerzahl erreichte Grenze 25.
| command: /home/reuter/local/lib/whizard/libtool --mode=compile gfortran -c -I/home/reuter/local/lib/whizard/mod/models -I/home/reuter/local/include/omega -I/home/reuter/local/lib/whizard/mod/whizard-core -I/home/reuter/local/lib/whizard/mod/misc  -g -O2 'fail.f90'
| Return code = 256
******************************************************************************
******************************************************************************
*** FATAL ERROR: System command returned with nonzero status code
******************************************************************************
******************************************************************************

[kilian]

Compile with -ffree-line-length-none.

It is sufficient to write in the SINDARIN file

$fflags = $fflags & " -ffree-line-length-none"

or similar.

Should be a FAQ, Would be nice if O'Mega didn't produce lines > 132 chars, but that might be unavoidable. ThO, can you comment?

[ohl]

Replying to kilian:

ThO, can you comment?

It's a trivial tring and I'm currently testing a fix.

Explanation: I'm appending " &" to lines that are continued and I did not count these 2 characters in the overall line length. Thus if O'Mega is called with -target:width 132, there can be (in very rare cases) lines with 133 or 134 characters, violating the Fortran standard. It just never appeared before ...

Now I'm subtracting 2 from the argument of -target:width and everything should be fine.

[ohl]

Fixed in #2588.

[Juergen Reuter]

The problem is still there, at least I found it on 32bit Linux, try the process:

alias pr = u:d:s:c:U:D:S:C:g
alias jet = u:d:s:c:U:D:S:C:g
alias q  = u:d:s:c
alias Q  = U:D:S:C
alias lep = e1:E1
alias neu = n1:N1
process wbb_16c = Q, g => lep,neu,b,B,Q,q,Q

On a 64bit machine that worked. On the 32bit machine I unfortunately do not have the 4.5.0 release compiler, but a pre-version, but I guess that does not make a difference.

[ohl]

32bit address space issues are not invalid O'Mega Code.

The O'Mega part of this is fixed as of r2588. I just confirmed this on 64bit Debian. 32bit address space issues and not invalid O'Mega Code.

[Juergen Reuter]

Well, it was not a 32 bit issue. It was just the fact that there were two versions of WHIZARD installed on the machine, and the default O'Mega binaries were still the old ones. :-( Deh non volerli vittime del mio fatale errore!

[sschmidt]

A similar defect: If compiled with nagfor the process wbb_16c (see above) fails with the error message

Fatal Error: wbb_16c.f90, line 3062: Too many continuation lines
| command: /localscratch/sschmidt/whizard/2.00_nagfor/lib/whizard/libtool --mode=compile nagfor -c -I/localscratch/sschmidt/whizard/2.00_nagfor/lib/whizard/mod/models -I/localscratch/sschmidt/whizard/2.00_nagfor/include/omega -I/localscratch/sschmidt/whizard/2.00_nagfor/lib/whizard/mod/whizard-core -I/localscratch/sschmidt/whizard/2.00_nagfor/lib/whizard/mod/misc  -g90 'wbb_16c.f90'
| Return code = 256
******************************************************************************
******************************************************************************
*** FATAL ERROR: System command returned with nonzero status code
******************************************************************************
******************************************************************************

because the Fortran 2003 standard allows a maximum of 255 continuation lines, while in wbb_16c.f90 there is a type(spinor) declaration that is ~700 lines long.

[ohl]

Fair enough.

That's easy to fix. But soon I'll have to break up the color flow table as well ...

[ohl]

r2592 limits the # of declarations per statements. Chopping up of color flow tables will come.

[ohl]

Since JR has opened #348, this one is redundant.

[Juergen Reuter]

Yep, the color flow table part of this is redundant.

[Juergen Reuter]

The problem that the code gets too large for large processes due to limitations of the address space is still persistent. Here are CS comments (in German):

1) Erhaltene Quantenzahlen pruefen, um verbotene Prozesse sofort auszusortieren (ohne einen DAG zu berechnen) - Modelle koennten einen Typ QuantumNumbers? mit Addition, Subtraktion und Gleichheit sowie eine Abbildung von Flavor auf QuantumNumber? definieren. Dies koennte die Laufzeit von OMega fuer viele Tensorprodukte im Endzustand drastisch reduzieren.

2) Identische Prozesse identifizieren und Code nur einmal generieren (und auch nur einmal mit einem Faktor n verziert berechnen). Dafuer koennten Modelle eine Funktion definieren, welche testet, ob zwei Flavor als identisch anzusehen sind. Dies koennte sowohl die Laufzeit von OMega und die Codegroesse reduzieren als auch die Geschwindigkeit des Codes erhoehen.

3) Die Information ueber identische Flavor in der Cross-Flavor-CSE beruecksichtigen. Damit koennten identische Beitraege zu g g -> u ubar c cbar und g g -> u ubar u ubar nur einmal berechnet werden, was die Codegroesse reduziert und die Laufgeschwindigkeit erhoeht.

4) Rotationsymmetrie ausnutzen: die Prozesse u g -> X und g u -> X unterscheiden sich nur um eine Rotation um die x- (oder y-) Achse. Dadurch koennte man fuer Hadronprozesse bis zu 50 Prozent an Codegroesse sparen. Allerdings muesste man das fuer Modelle mit externen Vektorfeldern deaktivieren koennen.

5) Crossing-Symmetrie ausnutzen: damit muesste z.B. fuer g g -> u ubar, g u -> g u und g ubar -> g ubar nur einmal Code erzeugt werden, was die Codegroesse reduziert.

Auf diese Art und Weise sollte es moeglich sein, die Groesse des erzeugten Codes erheblich zu reduzieren (ich koennte mir durchaus eine Groessenordnung vorstellen), was zumindest fuer 2 -> 8 die Addressraumprobleme loesen koennte. Ausserdem wuerde insbesondere 3) die Geschwindigkeit des Codes deutlich erhoehen. Allerdings muesste fuer 2) und 3) eine Moeglichkeit bestehen, OMega Information ueber verschwindende Massen zukommen zu lassen; das ist aber ohnehin fuer die analytischen Helizitaetsauswahlregeln notwendig.

1) is done, the rest not (yet).

[ohl]

5) is #352.

The other points are also on my agenda, but I wanted to have separate tickets, because it's not possible to close sub-tickets.

[Juergen Reuter]

There is some other issue: for a certain process for ttbar + 1 Jet (i.e. q,g -> b bbar l+ l- nu nubar q for q = u,d,c,s,U,D,C,S already the phase space generation is big enough to trigger the "out of memory" failure!

[ohl]

see #364