Difference between revisions of "GetDDM"

Latest revision as of 12:20, 13 May 2017

GetDDM combines GetDP and Gmsh to solve large scale finite element problems using optimized Schwarz domain decomposition methods.

Examples

• Non-overlapping DDM for time-harmonic waves (Helmholtz and Maxwell)

Precompiled binaries

For demonstration purposes, download the serial pre-compiled ONELAB bundle for Windows (32 bit), Linux or MacOS. With these precompiled binaries the examples will run in sequential mode. For parallel computations you need to recompile the codes from source with MPI support (see below).

Parallel version build

For parallel computations you need to compile GetDP and Gmsh with MPI support.

• Install MPI and CMake.

• Download PETSc from http://www.mcs.anl.gov/petsc/petsc-as/download/. (PETSc 3.4, 3.5, 3.6 and 3.7 have been tested.)

• Uncompress the PETSc archive (in this example, using PETSc 3.7.5):

tar zxvf petsc-3.7.5.tar.gz

• Configure and build PETSc. The configuration options depend on the calculations you want to perform (complex- or real-valued), as well as your compiler/MPI/Blas/Lapack setup. Make sure that mpicc, mpicxx and mpif90 are in your path. Then run:

cd petsc-3.7.5
export PETSC_DIR=$PWD
export PETSC_ARCH=mpi_mumps_complex
# Notes:
# * Remove option --with-scalar-type=complex to build in real arithmetic
# * Use option --download-fblaslapack=1 if you don't have optimized bias/lapack libraries available on your system
./configure --with-debugging=0 --with-clanguage=cxx --with-shared-libraries=0 --with-x=0  --download-mumps=1 --download-metis=1 --download-parmetis=1 --download-scalapack=1 --download-blacs=1 --with-scalar-type=complex
make
cd ..

• Download and unzip the Gmsh and the GetDP source code from the ONELAB bundle : http://onelab.info/files/gmsh-getdp-source.zip. Alternatively, you can download the last development version of Gmsh and GetDP from the source repositories. See http://getdp.info and http://gmsh.info for more information.

svn co https://onelab.info/svn/gmsh/trunk gmsh --username gmsh # the password is the same as the username
svn co https://onelab.info/svn/getdp/trunk getdp --username getdp # the password is the same as the username

• Configure, compile and install a minimal Gmsh library (it will be used by GetDP):

cd gmsh-xxx
mkdir lib
cd lib
cmake -DDEFAULT=0 -DENABLE_PARSER=1 -DENABLE_POST=1 -DENABLE_BUILD_LIB=1 ..
make lib
sudo make install/fast # or 'make DESTDIR=$HOME/install-path-gmsh/ install/fast' if you do not have root access
cd ../..

• Configure and compile the MPI version of GetDP (change CC, CXX and FC depending on your MPI installation):

cd getdp-xxx
mkdir bin
cd bin
cmake -DENABLE_MPI=1 ..
# Notes :
# * use option -DCMAKE_PREFIX_PATH=non-standard-install-path if you libraries installed in non-standard locations
# * use options -DPETSC_DIR=... -DPETSC_ARCH=... if the corresponding environment variables are not set properly
make
cd ../..

• [Optional - only for parallel mesh generation] Configure and compile the MPI version of Gmsh (need to disable Metis due to version clash with PETSc):

cd gmsh
mkdir bin
cd bin
cmake -DENABLE_MPI=1 -DENABLE_METIS=0 ..
make

Parallel runs

The commands for running GetDDM in parallel will depend on your particular MPI setup. Basically, the procedure would look something like this on 100 CPUs:

mpirun -np 100 gmsh file.geo -
mpirun -np 100 getdp file.pro -solve DDM 

Sample scripts for SLURM and PBS schedulers are available: [1], [2].

References

• C. Geuzaine, B. Thierry, N. Marsic, D. Colignon, A. Vion, S. Tournier, Y. Boubendir, M. El Bouajaji, and X. Antoine. An Open Source Domain Decomposition Solver for Time-Harmonic Electromagnetic Wave Problems. 2014 IEEE International Conference on Antenna Measurements & Applications. November 16-19, Antibes Juan-les-Pins, France.

• B. Thierry, A.Vion, S. Tournier, M. El Bouajaji, D. Colignon, N. Marsic, X. Antoine, C. Geuzaine. GetDDM: an Open Framework for Testing Optimized Schwarz Methods for Time-Harmonic Wave Problems. 2015.