GSI Forum
GSI Helmholtzzentrum für Schwerionenforschung

Home » PANDA » PandaRoot » Bugs, Fixes, Releases » new tool in PandaRoot: mpiTools
new tool in PandaRoot: mpiTools [message #6142] Sat, 22 March 2008 13:52 Go to next message
Johan Messchendorp is currently offline  Johan Messchendorp
Messages: 693
Registered: April 2007
Location: University of Groningen
first-grade participant

From: *xs4all.nl
Dear all,

Some time ago, I added a new package in the toolbox of PandaRoot (/PndTools) called "mpiTools". This is a small program exploiting MPI (message passing interface) which allows an "easy" and a parallel distribution of jobs on computing clusters such as PBS/LSF or any set of computers which can communicate via ssh or rsh. It can, furthermore, run on shared-memory systems, such as multi-core machines. This code is based on a boss-workers model. More detailed information can be found in a README file in the corresponding directory. Here you will also find example scripts and macros.

Kind wishes and happy Easter,

Johan



Johan Messchendorp
University of Groningen/KVI
Zernikelaan 25
NL-9747 AA Groningen
The Netherlands
tel. +31-503633558
fax +31-503634003

[Updated on: Sun, 21 December 2008 12:31]

Report message to a moderator

Re: new tool in PandaRoot: mpiTools [message #6668 is a reply to message #6142] Mon, 05 May 2008 20:39 Go to previous messageGo to next message
Johan Messchendorp is currently offline  Johan Messchendorp
Messages: 693
Registered: April 2007
Location: University of Groningen
first-grade participant

From: *xs4all.nl
Hi all,

For those interested. I made some significant updates of the MPI tools. Checkout the README file in ../pandaroot/PndTools/mpiTools/ to see what this tool can do for you.

Some of the updates are:

* Optimized boss-worker handshaking mechanism
* Improved error handling: failure results are send back to boss process, option to do a validation of the results
* A more flexible job description file with the option of user-defined parameters (dynamic)
* Added the option to download input data (remote or local) prior to the job
* execution scripts can be stored remotely (copied automatically)
* Bug fixes and code tested/debugged with MPICH and OPENMPI on PBS cluster in Groningen and on various multi-core platforms.
* Flexible and improved run numbering
* Added extensive example scripts with validation
* Added statistics output (run time)
* Job merging improved by using a BARRIER synchronization mechanism. Example scripts are included in the repository.
* Checks on disk storage and number of processes still pending with automatic hold


Johan.
Re: new tool in PandaRoot: mpiTools [message #6742 is a reply to message #6142] Sat, 17 May 2008 13:28 Go to previous messageGo to next message
Johan Messchendorp is currently offline  Johan Messchendorp
Messages: 693
Registered: April 2007
Location: University of Groningen
first-grade participant

From: *xs4all.nl
Dear all,

The "HOWTO" documentation of the MPI tools can be found at

http://panda-wiki.gsi.de/cgi-bin/viewauth/Computing/PandaRootTools


Johan.
Re: new tool in PandaRoot: mpiTools [message #7654 is a reply to message #6142] Sun, 21 December 2008 12:28 Go to previous message
Johan Messchendorp is currently offline  Johan Messchendorp
Messages: 693
Registered: April 2007
Location: University of Groningen
first-grade participant

From: *xs4all.nl
Hi,

I added a few new features (downwards compatible, though) in mpiTools, namely

* more options can be set "on-the-fly" via the job description file, such as timeouts, scratch path, minimum required disk space, maximum allowed threads per worker, etc. With this, one does not need to specify these options on the start-up of the code itself.
* it is possible to specify different TIMEOUTS for the time period a job is being killed and the sleep period in case a worker suffers from insufficient disk space or too many thread already running. Before, only one TIMEOUT could be specified.
* An option "NICE <lvl>" can be specified in the job description file, which sets the priority level of the following JOBS, e.g. the following jobs will run with a nice level of <lvl> on the worker nodes. Note that -20 is the highest priority and +20, the lowest. Negative nice levels can only be set by the root user.
* An option "MAXLOAD <lvl> can be specified in the job description file, which defines the maximum load of a worker node. Exceeding this number will put the worker to sleep specified in TIMEOUTS.
* Added a wrapper script "runmpi" for an MPI session, which is in particularly interesting for heterogeneous systems
* Several optimizations, better error and log handling
* Inclusion of CPU time in log output
* PBS script modified to support MPI2

The wiki site http://panda-wiki.gsi.de/cgi-bin/view/Computing/PandaRootTools and the README file have been updated according to the changes. The updates have been carried out in the "trunk" branch of PandaRoot (rev4257 or higher).

Johan.

[Updated on: Wed, 31 December 2008 01:40]

Report message to a moderator

Previous Topic: problem when running statistics larger than 2 000 events
Next Topic: problem with external packages installation
Goto Forum:
  


Current Time: Sun Dec 08 23:30:59 CET 2024

Total time taken to generate the page: 0.00620 seconds