Trade paths in system_dependencies for user variables

[Juergen Reuter]

Hardcoded paths should be avoided for several reasons. There should also be a possibility for adding extra libraries etc. in process compilation.

[Juergen Reuter]

I'm not sure whether I understand this ticket 100 per cent. It is clear that hard-coded paths should be avoided. But how exactly? It cannot be that the user always has to specify paths in detail when using WHIZARD.

[kilian]

Looking in the RPM documentation (as a sample case where this matters), I would conclude that the best way is to depend on standard variables $prefix, $exec_prefix etc. The setting as it is now refers to the final installation path, this is fine. However, the path variables should be available as SINDARIN variables and/or command line options.

[kilian]

r2072: WHIZARD now understands and expands ${prefix}, ${exec_prefix}, ${bindir}, ${libdir}, ${includedir}, and ${datarootdir} in path variables. The paths are still hardcoded, but these GNU standard path variables can be reset on the command line, e.g.:

whizard --prefix=/my/own/whizard/installation

and any path that contains ${prefix} will be appropriately reset.

This is useful if the whizard installation is not in the directory where configure was told it should be. Normally, it should be irrelevant.

[Juergen Reuter]

Milestone v2.0-rc4 deleted

[Boschmann]

When I run

./configure --prefix=/my/own/whizard/installation
make install
cd /my/own/whizard/installation/bin
./whizard --prefix=/my/own/whizard/installation

then whizard complains:

./whizard: error while loading shared libraries: libwhizard_main.so.0: cannot open shared object file: No such file or directory

The libraries of whizard are distributed to several directories, so you cannot repair this easily by setting LD_LIBRARY_PATH.

./whizard -h

also doesnt't work without libwhizard_main.so.0. It would be nice to get the help information to correctly set up the environment variables.

[Juergen Reuter]

Can anyone comment on this ticket pls!? For sure what HWB describes cannot work. I would close this as duplicate to the other tickets.... (?)

[Boschmann]

This is what I do in jenkins:

for x in circe1 circe2 omega omega/caml vamp whizard
do
LD_LIBRARY_PATH=$WORKSPACE/$WHIZARD/$x:$LD_LIBRARY_PATH
done
export LD_LIBRARY_PATH
export PATH=$WORKSPACE/$WHIZARD/bin:$PATH
$WORKSPACE/$WHIZARD/bin/whizard --prefix=$WORKSPACE/$WHIZARD $SIN_FILE

I can create a shell wrapper in this manner and add it to the whizard trunk. Users should still use the binary directly, but with this wrapper people *can* relocate whizard and use the shell script instead. This was the intended way to run "instand whizard", but as soon as such a wrapper exist, we should ship it with the distribution tarball.

[kilian]

Setting paths: Isn't that the purpose of whizard-setup.sh in bin? What I don't get: Do you want to make an uninstalled whizard package run, or an installed one?

[Boschmann]

I want to do:

./configure --prefix=/SOMWHERE
make
make install
mv /SOMWHERE /SOMEWHERE_ELSE
/SOMEWHERE_ELSE/bin/whizard --prefix=/SOMEWHERE_ELSE

[Boschmann]

This is what I get: (after inserting a missing /lib in line 3)

libtool: compile:  gfortran -c -I/SOMWHERE_ELSE/lib/whizard/mod/models -I/SOMWHERE_ELSE/include/omega -I/SOMWHERE_ELSE/lib/whizard/mod/whizard-core -I/SOMWHERE_ELSE/lib/whizard/mod/misc eeww.f90  -fPIC -o .libs/eeww.o
libtool: compile:  gfortran -c -I/SOMWHERE_ELSE/lib/whizard/mod/models -I/SOMWHERE_ELSE/include/omega -I/SOMWHERE_ELSE/lib/whizard/mod/whizard-core -I/SOMWHERE_ELSE/lib/whizard/mod/misc eeww.f90 -o eeww.o >/dev/null 2>&1
libtool: compile:  gfortran -c -I/SOMWHERE_ELSE/lib/whizard/mod/models -I/SOMWHERE_ELSE/include/omega -I/SOMWHERE_ELSE/lib/whizard/mod/whizard-core -I/SOMWHERE_ELSE/lib/whizard/mod/misc processes.f90  -fPIC -o .libs/processes.o
libtool: compile:  gfortran -c -I/SOMWHERE_ELSE/lib/whizard/mod/models -I/SOMWHERE_ELSE/include/omega -I/SOMWHERE_ELSE/lib/whizard/mod/whizard-core -I/SOMWHERE_ELSE/lib/whizard/mod/misc processes.f90 -o processes.o >/dev/null 2>&1
libtool: link: warning: library `/SOMWHERE_ELSE/lib/whizard/libwhizard_main.la' was moved.
libtool: link: warning: library `/SOMWHERE_ELSE/lib/whizard/libomega.la' was moved.
/bin/grep: /SOMEWHERE/lib/omega/libomega_core.la: No such file or directory
/bin/sed: can't read /SOMEWHERE/lib/omega/libomega_core.la: No such file or directory
libtool: link: `/SOMEWHERE/lib/omega/libomega_core.la' is not a valid libtool archive
| command: /SOMWHERE_ELSE/lib/whizard/libtool --mode=link gfortran -module -rpath /usr/local/lib    -L/SOMWHERE_ELSE/lib/whizard -lwhizard_main -lwhizard -lomega     -lomega -L/SOMWHERE_ELSE/lib/whizard/models -lwhizard -o 'processes.la'  eeww.lo processes.lo
| Return code = 256
******************************************************************************
******************************************************************************
*** FATAL ERROR: System command returned with nonzero status code
******************************************************************************
******************************************************************************
WHIZARD run aborted.
Finished: FAILURE

[Juergen Reuter]

Should this be possible at all?? Sorry, somehow I lost track of all the tickets with DESTDIR, install, prefix etc. :((((

[Christian Speckner]

I don't see the point of this discussion. The intented use of the distro download is: configure - compile - install - use. In this case, no problems arise. Disributors and package builders can use the variation configure - compile - install to staging area via DESTDIR - deploy - use which works as well. Only for the very special (instant WHIZARD) case of relocating the installation prefix, additional tricks like more elaborate LD_LIBRARY_PATH settings and addition WHIZARD options are required, but those can be easily done in a wrapper script. Of course it is possible to ship this script with the distro, but it is not necessary if the distro is used in the first place anyway.

[Boschmann]

Imagine you want to run whizard on a cluster with gcc 3.4. The owner doesn't allow you to install software to system directories, all you've got is a temporary folder somewhere on the grid. Now imagine, you don't want to run a single huge job but a large number of medium jobs, that's why you use a batch system. Now what? Would you pack all sources of gmp, mpfr, mpc, gcc, ocaml, lhapdf and whizard into a tarball and run ./configure --prefix=. ; make ; make install for each job?

Here again instand-whizard comes in handy and this is actually the way I stumbled upon this problem at all. Jenkins is a batch system and I want to assign a job to each example sindarin file, but I don't want to recompile whizard for each job. We also want to run the "tables" to check some reference cross sections periodically on the latest whizard.

So the relocatable issue is not that notional as it might look at the first sight. Other users might face the same problem with cluster access and this is why I suggested to ship the wrapper with whizard: Users can recompile whizard with any additional flags or libs and upload the binaries to the cluster together with the wrapper.

[kilian]

Fully agree. One idea is to use a static build of Whizard for batch systems (see #365), which takes care of the LD_LIBRARY_PATH issue. I'll see what to do with LHAPDF, this might also not be installed.

Either one wants the full functionality of Whizard on the batch job, this would imply to pack gcc and friends together with whizard, and might require dynamic libs. Or the processes are compiled in the executable (on a machine where gfortran 4.5 is present); in that case the Whizard binary is sufficient. In either case, one needs relocation for some additional files.

Let's see what can be done.

[Juergen Reuter]

It seems that WK wants to have this ready for 2.0.5. already.

[kilian]

With a few additions and extra scripts (r3089), WHIZARD appears to be fully relocatable.

• To make whizard launch successfully, LD_LIBRARY_PATH must cover all shared libs
• The fortran executable and lib and the stdc++ lib must be accessible
• lhapdf must be accessible, also HepMC
• After relocation of the complete installation, libtool-relocate.sh can update the headers
• libtool itself may need target-specific configuration. libtool-config.sh does that.
• Gamelan now accepts arguments for relocation
• whizard has the --prefix option, also a --libtool option if libtool was reconfigured.

This is for the full dynamic configuration. For the static executable, this is simpler. One needs only the Fortran and C++ libraries in an accessible path.