recompilation de solver-bench sur judith (CentOS7)
dune sur judith, ne pas oublier:
[root@judith ~]# yum groupinstall 'Development Tools'
- dans /local/apps/src/dune
- dernière version du solver-bench: 0.70
- a besoin de : (voir config.log)
--with-umfpack=/local/apps/src/dune/UMFPACK --with-petsc=/local/apps/src/dune/petsc-3.1-p5 --with-dune-istl=/local/apps/src/dune/Dune --with-slepc=/local/apps/src/dune/slepc-3.1-p4/linux-gnu-c-opt
si on essaie de recompiler UMFPACK en utilisant le script de RK: ./umfpack_installer.sh
./umfpack_simple: error while loading shared libraries: libgfortran.so.1: cannot open shared object file: No such file or directory
et:
[root@judith ~]# yum provides "*/libgfortran.so.1"
compat-libgfortran-41-4.1.2-44.el7.i686 : Compatibility Fortran 95 runtime
: library version 4.1.2
Repo : base
Matched from:
Filename : /usr/lib/libgfortran.so.1
on installe cette lib:
[root@judith ~]# yum install compat-libgfortran-41-4.1.2-44.el7
et on lance l'install, voir umfpack_installer.LOG, qui montre plein de différences sur les résultats obtenus, est ce que c'est génant?
autre erreur, le do_work.sh ne passe pas à cause du compilo fortran qui n'accepte plus le -R alors qu'avant, il signalait juste l'erreur mais créeait quand même l'exécutable. On enlève le -R de LD_FLAGS
et ça passe:
Found the following Dune-components: ---------------------------------------- dune-common.............: yes (/local/apps/src/dune/Dune) version 2.0 dune-istl...............: yes (/local/apps/src/dune/Dune) version 2.0 BLAS....................: yes METIS...................: no MPI.....................: no PETSC...................: yes (/local/apps/src/dune/petsc-3.1-p5) ParMETIS................: no SLEPC...................: yes (/local/apps/src/dune/slepc-3.1-p4/linux-gnu-c-opt) SuperLU-DIST............: no SuperLU.................: no UMFPACK.................: yes
ensuite, on lance le make
autre erreur:
/local/apps/src/dune/Dune/include/dune/common/nullptr.hh:25:24: error: conflicting declaration ‘typedef class dune_nullptr_t nullptr_t’
typedef dune_nullptr_t nullptr_t;
^
In file included from /local/apps/src/dune/UMFPACK/AMD/Include/amd.h:45:0,
from /local/apps/src/dune/UMFPACK/UMFPACK/Include/umfpack.h:91,
from umfpack.hh:34,
from mymatrix.hh:78,
from benchmarkruntime.cc:35:
/usr/lib/gcc/x86_64-redhat-linux/4.8.2/include/stddef.h:432:29: error: ‘nullptr_t’ has a previous declaration as ‘typedefstd::nullptr_t nullptr_t’
typedef decltype(nullptr) nullptr_t;
^
on va essayer de commenter le typedef dans /local/apps/src/dune/Dune/include/dune/common/nullptr.hh on va ajouter -DHAVE_NULLPTR au compilo
ça passe, mais il y a un autre problème à l'édition de liens du à petsc qui n'a pas été recompilé
on essaie de recompiler petsc, et RK a fourni le script install_petsc_slepc.sh avec all comme argument ça passe toujours pas, on essaie autrement (voir wiki)
bash-4.2$ export PETSC_DIR=$PWD bash-4.2$ ./config/configure.py --with-cc=gcc --with-fc=gfortran --with-mpi=0 ... Compilers: C Compiler: gcc -Wall -Wwrite-strings -Wno-strict-aliasing -g3 Fortran Compiler: gfortran -Wall -Wno-unused-variable -g Linkers: Static linker: /bin/ar cr Dynamic linker: /bin/ar X11: Includes: Library: -lX11 BLAS/LAPACK: -llapack -lblas PETSc: PETSC_ARCH: linux-gnu-c-debug PETSC_DIR: /local/apps/src/dune/petsc-3.1-p5 Clanguage: C Scalar type: real Precision: double Memory alignment: 16 shared libraries: disabled dynamic libraries: disabled xxx=========================================================================xxx Configure stage complete. Now build PETSc libraries with: make PETSC_DIR=/local/apps/src/dune/petsc-3.1-p5 PETSC_ARCH=linux-gnu-c-debug all xxx=========================================================================xxx
il faut aussi ajouter la lib mpi et Different compiler and linker combinations require different flags. -R works for many GNU compiler and linker combinations, but not all compilers and linkers. In some cases, you may have to instead tell the compiler to pass the -rpath flag to the linker. One typical way to do this is with -Wl,-rpath,/usr/local/lib. dans LD_FLAGS dans do_work.sh pour compiler solver-bench, il a fallu aussi lancer certaines lignes du makefile à la main, ce qui est passé alors que depuis le makefile non
libtool: link: gfortran -g -O2 -Wl,-rpath -Wl,/usr/lib64/openmpi/lib --tag=CXX -Wl,-rpath -Wl,/usr/lib64/openmpi/lib -o fortrantest example_F.o writexdr.o -Wl,-rpath -Wl,/usr/lib64/openmpi/lib -L/local/apps/src/dune/petsc-3.1-p5/linux-gnu-c-opt/lib -L/usr/lib64/openmpi/lib -L/local/apps/src/dune/slepc-3.1-p4/linux-gnu-c-opt/lib -L/local/apps/src/dune/petsc-3.1-p5/lib -L/local/apps/src/dune/UMFPACK/UMFPACK/Lib -L/local/apps/src/dune/UMFPACK/AMD/Lib -L/local/apps/src/dune/Dune/lib -lslepc -lpetsc /local/apps/src/dune/Dune/lib/libdunecommon.a -lumfpack -lamd -lm -lmpi -llapack -lblas gfortran: error: unrecognized command line option '--tag=CXX' make[2]: *** [fortrantest] Error 1 make[2]: Leaving directory `/local/apps/src/dune/solver-bench-0.70/examples'
il faut aller dans le Makefile du dossier examples/ et virer –tag=CXX des lignes LD…., avec la nouvelle verson du compilo gfortran ça passe sans problème
on vérifie sur un des benches:
--------------------------------------------------------- RESULTS for `LS-FVM_test1_dkershaw16.xdr' --------------------------------------------------------- Contributor and scheme: LS-FVM Test: 1 Mesh: dkershaw16 --------------------------------------------------------- Problem size: 1) unknowns: 4096 2) non-zero Matrix entries: 97336 3) entries in blocks of size: 1 Solver + Preconditioner: ISTL-CG + ILU (0) (relax. 1) Reduction to achieve: 1e-10 Number of iterations: 409601 Residual : || b - Ax ||_inf = 2.04981 rel. Res.: || b - Ax ||_inf / || b ||_inf = 2.32316 Computed eigenvalues values: lambda_max = -11.2098, lambda_min = -0.00820027 Estimated condition number: lambda_max/lambda_min = 1367 CPU time to solve in seconds: 101.545 Memory used in MB: 4.74609 (program internal) Memory used in MB: 7.9375 (rusage.ru_maxrss) ---------------------------------------------------------
à comparer avec http://solqual.latp.univ-provence.fr/ref_bench/a5fe7a76a75737a80d2c42e7d64b68e3/, qui montre qu'on a gagné un facteur 3!
| CPU time to solve in seconds | 369.315 | 101.545 |
installation sur octopus
[devel1@octopus ~/src]$ tar xzf ../downloads/dune-common-2.0.tar.gz mkdir dune cd dune [devel1@octopus ~/src]$ ./dune-common-2.0/bin/dunecontrol all [devel1@octopus dune]$ ./dune-common-2.0/bin/dunecontrol --opts=config.opts all
installer une nouvelle version de solver-bench
- dans
/local/apps/src/dune - déplacer l'ancienne version
- récuperer le fichier do_work.sh
- ne pas oublier de rajouter -lmpi dans LIBS (voir ci-dessous)
benchmark
- R.K a installé /local/apps/src/dune/solver-bench-0.1beta
- recopié ce dossier dans /home/henry/projects/
- dans solver-bench-0.1beta/src, il y a un programme de bench :
[henry@octopus src]$ ./benchruntime Usage: ./benchruntime <datafile.xdr> < UMFPACK = 0 | ISTL-CG = 1 | ISTL-BiCGstab = 2 | OEM-CG = 3 | OEM-GMRES = 4 > Residual: euclidian = 1.21829e-29 infinity = 1.77636e-15 ERROR: UMFPack not available, re-configure with --with-umfpack=PATH_TO_UMFPACK!! UMFPACK: Final errors: euclidian = 0.493997 infinity = 0.561107 Solution needed 0 seconds.
avec un autre solveur et une autre matrice:
[henry@octopus src]$ ./benchruntime ../../herbin/AKWAK/APPS/TROISD/resultats/matrixfile_1 2
Residual: euclidian = 2.43625e-17 infinity = 3.3555e-09
Reduction to achieve: 3.10816e-19
=== BiCGSTABSolver
Iter Defect Rate
0 8.85338
0.5 4.34976 0.491311
1 0.477752 0.109834
1.5 0.328951 0.688539
2 0.00710675 0.0216043
2.5 0.00885003 1.2453
3 0.00118422 0.13381
3.5 0.000483821 0.408555
4 2.3323e-05 0.0482058
4.5 4.16061e-06 0.178391
5 1.17866e-06 0.283291
5.5 7.05524e-07 0.59858
6 7.83043e-09 0.0110987
6.5 6.49605e-10 0.082959
7 9.89328e-11 0.152297
7.5 1.2981e-11 0.13121
8 1.51537e-12 0.116738
8.5 1.62378e-13 0.107154
9 3.17988e-14 0.195832
9.5 1.81317e-15 0.0570202
10 2.7893e-16 0.153835
10.5 1.04694e-17 0.0375342
11 2.14529e-18 0.20491
=== rate=0.0203082, T=0, TIT=0, IT=11
Difference : 2.91461e-09
ISTL-BiCGstab: Final errors: euclidian = 8.49496e-18 infinity = 1.88764e-09
Solution needed 0 seconds.
avec umfpack
- ajouter umfpack avec le script fourni par R.K. : umfpack_installer.sh
- umfpack s'installe dans le répertoire courant: /local/apps/src/dune/UMFPACK, en version 5.4.0
- on recompile dune et le module solver-bench avec umfpack
- on vérifie que c'est pris en compte:
[henry@octopus src]$ ./benchruntime ../../herbin/AKWAK/APPS/TROISD/resultats/matrixfile_1 0 Residual: euclidian = 2.43625e-17 infinity = 3.3555e-09 nR = 93 nC = 93 nZ = 1467 Rows offset is 0 Overall time for UMFPACK solve: 0.001 UMFPACK: Final errors: euclidian = 17.687 infinity = 0.996845 Solution needed 0 seconds.
petsc
[devel1@octopus petsc-3.1-p4]$ export PETSC_DIR=$PWD
[devel1@octopus petsc-3.1-p4]$ ./config/configure.py --with-cc=gcc --with-fc=gfortran
...
Compilers:
C Compiler: gcc -Wall -Wwrite-strings -Wno-strict-aliasing -g3
Fortran Compiler: gfortran -Wall -Wno-unused-variable -g
Linkers:
Static linker: /usr/bin/ar cr
X11:
Includes:
Library: -lX11
BLAS/LAPACK: -llapack -lblas
PETSc:
PETSC_ARCH: linux-gnu-c-debug
PETSC_DIR: /local/apps/src/petsc-3.1-p4
Clanguage: C
Scalar type: real
Precision: double
Memory alignment: 16
shared libraries: disabled
dynamic libraries: disabled
xxx=========================================================================xxx
Configure stage complete. Now build PETSc libraries with:
make PETSC_DIR=/local/apps/src/petsc-3.1-p4 PETSC_ARCH=linux-gnu-c-debug all
xxx=========================================================================xxx
[devel1@octopus petsc-3.1-p4]$
[devel1@octopus petsc-3.1-p4]$ make PETSC_DIR=/local/apps/src/petsc-3.1-p4 PETSC_ARCH=linux-gnu-c-debug test
Running test examples to verify correct installation
C/C++ example src/snes/examples/tutorials/ex19 run successfully with 1 MPI process
Fortran example src/snes/examples/tutorials/ex5f run successfully with 1 MPI process
Completed test examples
nouvelle version de solver-bench (0.45) plus slepc
L'exemple ci-dessous est avec le mode debug, voir les fichiers do_work.sh pour le code optimisé
bash-3.2$ ./config/configure.py --with-mpi=0 bash-3.2$ make PETSC_DIR=/local/apps/src/dune/petsc-3.1-p5 PETSC_ARCH=linux-gnu-c-debug all bash-3.2$ make PETSC_DIR=/local/apps/src/dune/petsc-3.1-p5 PETSC_ARCH=linux-gnu-c-debug test bash-3.2$ export PETSC_DIR=/local/apps/src/dune/petsc-3.1-p5 bash-3.2$ export PETSC_ARCH=linux-gnu-c-debug bash-3.2$ export SLEPC_DIR=/local/apps/src/dune/slepc-3.1-p4 bash-4.2$ cd ../slepc-3.1-p4 bash-3.2$ ./configure bash-3.2$ make bash-3.2$ make test Running test examples to verify correct installation C/C++ example src/examples/ex1 run successfully with 1 MPI process Fortran example src/examples/ex1f run successfully with 1 MPI process Completed test examples
après, on essaie:
bash-3.2$ ./installdune.sh solver-comp but i saw in the messages that petsc and slepc weren't found: Found the following Dune-components: ---------------------------------------- dune-common.............: yes (/local/apps/src/dune/Dune) version 2.0 dune-istl...............: yes (/local/apps/src/dune/Dune) version 2.0 BLAS....................: yes METIS...................: no MPI.....................: no PETSC...................: no ParMETIS................: no SLEPC...................: no SuperLU-DIST............: no SuperLU.................: no UMFPACK.................: yes (/local/apps/src/dune/UMFPACK) ---------------------------------------- See ./configure --help and config.log for reasons why a component wasn't found
si on édite config.log:
./configure CXX=g++ CXXFLAGS=-O3 -funroll-loops -finline-functions -DNDEBUG \ --prefix=/local/apps/src/dune/Dune \ --enable-parallel \ --disable-documentation --disable-compilercheck --disable-mpiruntest \ --with-umfpack=/local/apps/src/dune/UMFPACK \ --with-dune-common=/local/apps/src/dune/Dune --with-dune-istl=/local/apps/src/dune/Dune
alors que pour la derneire version:
./configure CXX=g++ CXXFLAGS="-O3 -funroll-loops -finline-functions -DNDEBUG" \ --prefix=/local/apps/src/dune/Dune \ --disable-documentation --disable-compilercheck --disable-mpiruntest \ --with-umfpack=/local/apps/src/dune/UMFPACK \ --with-petsc=/local/apps/src/petsc-3.1-p5 \ --with-dune-common=/local/apps/src/dune/Dune \ --with-dune-istl=/local/apps/src/dune/Dune \ CPPFLAGS="-I/local/apps/src/petsc-3.1-p5/linux-gnu-c-debug/include -I/usr/lib/openmpi/1.4-gcc/include" \ LDFLAGS="-L/local/apps/src/petsc-3.1-p5/linux-gnu-c-debug/lib"
et ca passe donc avec la ligne:
./configure CXX=g++ CXXFLAGS="-O3 -funroll-loops -finline-functions -DNDEBUG" --prefix=/local/apps/src/dune/Dune --disable-documentation --disable-compilercheck --disable-mpiruntest --with-umfpack=/local/apps/src/dune/UMFPACK --with-petsc=/local/apps/src/dune/petsc-3.1-p5 --with-dune-common=/local/apps/src/dune/Dune --with-dune-istl=/local/apps/src/dune/Dune CPPFLAGS="-I/local/apps/src/dune/petsc-3.1-p5/linux-gnu-c-debug/include -I/usr/lib/openmpi/1.4-gcc/include" LDFLAGS="-L/local/apps/src/dune/petsc-3.1-p5/linux-gnu-c-debug/lib"
ne pas oublier l'include openmpi, car il a besin de mpi.h alors qu'on a demande de compiler petsc sans mpi
pour rajouter slepc:
./configure CXX=g++ CXXFLAGS="-O3 -funroll-loops -finline-functions -DNDEBUG" \ --prefix=/local/apps/src/dune/Dune \ --disable-documentation --disable-compilercheck --disable-mpiruntest \ --with-umfpack=/local/apps/src/dune/UMFPACK \ --with-petsc=/local/apps/src/dune/petsc-3.1-p5 \ --with-dune-common=/local/apps/src/dune/Dune \ --with-dune-istl=/local/apps/src/dune/Dune \ --with-slepc=/local/apps/src/dune/slepc-3.1-p4/linux-gnu-c-debug \ --with-slepc-includedir=/local/apps/src/dune/slepc-3.1-p4/include \ --with-slepc-libdir=/local/apps/src/dune/slepc-3.1-p4/linux-gnu-c-debug/lib \ CPPFLAGS="-I/local/apps/src/dune/petsc-3.1-p5/linux-gnu-c-debug/include -I/usr/lib/openmpi/1.4-gcc/include -I/local/apps/src/dune/slepc-3.1-p4/include" \ LDFLAGS="-L/local/apps/src/dune/petsc-3.1-p5/linux-gnu-c-debug/lib -L/usr/lib64/openmpi/1.4-gcc/lib" \ LIBS="-lmpi -lX11" ... Found the following Dune-components: ---------------------------------------- dune-common.............: yes (/local/apps/src/dune/Dune) version 2.0 dune-istl...............: yes (/local/apps/src/dune/Dune) version 2.0 BLAS....................: yes METIS...................: no MPI.....................: no PETSC...................: yes (/local/apps/src/dune/petsc-3.1-p5) ParMETIS................: no SLEPC...................: yes (/local/apps/src/dune/slepc-3.1-p4/linux-gnu-c-debug) SuperLU-DIST............: no SuperLU.................: no UMFPACK.................: yes (/local/apps/src/dune/UMFPACK) ---------------------------------------- See ./configur
Attention, cette compilation échoue, il faut enlever LIBS, lancer le configure, et hacker manuellement les 2 Makefile pour rajouter ces libs!
bash-3.2$ diff /tmp/Makefile src/Makefile
225c225
< LIBS = -L/local/apps/src/dune/Dune/lib -ldunecommon -L/local/apps/src/dune/Dune/lib -ldunecommon ${DUNEMPILIBS} -L/local/apps/src/dune/Dune/lib -ldunecommon -L/local/apps/src/dune/Dune/lib -ldunecommon -lm -lumfpack -lamd -lpetsc -lslepc -llapack -lblas
---
> LIBS = -L/local/apps/src/dune/Dune/lib -ldunecommon -L/local/apps/src/dune/Dune/lib -ldunecommon ${DUNEMPILIBS} -L/local/apps/src/dune/Dune/lib -ldunecommon -L/local/apps/src/dune/Dune/lib -ldunecommon -lm -lumfpack -lamd -lpetsc -lslepc -llapack -lblas -lmpi