/*
 * Copyright (c) 2014:  G-CSC, Goethe University Frankfurt
 * Author: Martin Rupp
 * 
 * This file is part of UG4.
 * 
 * UG4 is free software: you can redistribute it and/or modify it under the
 * terms of the GNU Lesser General Public License version 3 (as published by the
 * Free Software Foundation) with the following additional attribution
 * requirements (according to LGPL/GPL v3 §7):
 * 
 * (1) The following notice must be displayed in the Appropriate Legal Notices
 * of covered and combined works: "Based on UG4 (www.ug4.org/license)".
 * 
 * (2) The following notice must be displayed at a prominent place in the
 * terminal output of covered works: "Based on UG4 (www.ug4.org/license)".
 * 
 * (3) The following bibliography is recommended for citation and must be
 * preserved in all covered files:
 * "Reiter, S., Vogel, A., Heppner, I., Rupp, M., and Wittum, G. A massively
 *   parallel geometric multigrid solver on hierarchically distributed grids.
 *   Computing and visualization in science 16, 4 (2013), 151-164"
 * "Vogel, A., Reiter, S., Rupp, M., Nägel, A., and Wittum, G. UG4 -- a novel
 *   flexible software system for simulating pde based models on high performance
 *   computers. Computing and visualization in science 16, 4 (2013), 165-179"
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
 * GNU Lesser General Public License for more details.
 */

/** \page pageUG4ParallelCesari Cesari

\section secCesari_Architecture Architecture
- \em Cesari is the bigger in-house cluster of the \em G-CSC.
- It consists of 26 nodes with 2 10-core processors each, resulting in 20 cores/node and <b>520 cores total</b>.
- Each Compute node has 32 GB RAM, which means when running one mpi threads per core (=20 per node), you get <b>1.6 GB RAM per thread</b>.
- You can also use hyperthreading (=2 mpi threads per core), resulting in 40 threads/node an 0.8 GB RAM per thread.<br>
- The log-in node is configured like a normal node (2x 10 cores), but has 64 GB of RAM.
- Job scheduling is done via \ref pageugsubmit  (auto-detect)

\section secCesari_Module Module System
- <tt>module avail</tt>: displays all modules available.  
- <tt>module list</tt>: displays all modules currently loaded.
- <tt>module load MODULENAME</tt>: Loads the module MODULENAME
- <tt>module unload MODULENAME</tt>: Unloads the module MODULENAME

\note Please note that you have to use that <tt>module load</tt> each time you log in, so it makes sense to put them into your .bashrc.

\section secCesari_Compiling Compiling ug4


\subsection secCesari_CompilingGCC GCC
  First you have to select a MPI version to run jobs in parallel:
    \verbatim
    module load openmpi-gcc
    \endverbatim
    then you can use the normal <tt>cmake ..</tt>.    
    
\subsection secCesari_CompilingLLVM LLVM
  LLVM/clang (recommended):
    \verbatim
    module load llvm
    module load openmpi-clang    
    \endverbatim
    To compile with the clang-compiler, use  
	\verbatim
	cmake -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang ..
	\endverbatim


\section secCesari_Jobs Running parallel Jobs
To start <b>Jobs on Cesari</b>, use \c ugsubmit /   \c uginfo / \c ugcancel . <br>
\ref pageugsubmit .<br> 
ugsubmit supports cesari without any configuration (auto-detect). By default it uses 20 threads/node (no hyperthreading). You can override that behaviour with the parameter -nppn.


\section secCesari_Debugging Debugging
Debugging : DDT can be used via <tt>module load ddt</tt>



*/