spataro@briareos:~/jan12/pandaroot/macro/global$ rt tracks_BARREL_1000sep.C
root [0]
Processing tracks_BARREL_1000sep.C...
FairRootManager::OpenOutFile("tracks_22Part_n1000.root")
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [861.6 MB/s]
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
**** merged file = " /home/spataro/jan12/pandaroot/macro/global/allParams_20120412_92051.root "
[INFO ] The input consists out of the following trees and files:
[INFO ] - cbmsim
[INFO ] - /home/spataro/jan12/pandaroot/macro/global/points_22Part_n1000_f0_sep.ro ot
FairRunAna::RunOnProof(0,0): running FairAnaSelector on proof server: "" with PAR file name = "$VMCWORKDIR/gconfig/libFairRoot.par".
+++++++ T P R O O F +++++++++++++++++++++++++++++++++
creating TProof* proof = TProof::Open("");
+++ Starting PROOF-Lite with 8 workers +++
Opening connections to workers: OK (8 workers)
Setting up worker servers: OK (8 workers)
PROOF set to parallel mode (8 workers)
+++++++ C R E A T E D +++++++++++++++++++++++++++++++
0309: ClearPackages
0309: UploadPackages
0309: EnablePackages
EXECUTING libFairRoot.par/SETUP.C without includes
Function SETUP_c7827247() busy. loaded after "/home/spataro/jan12/pandaroot/gconfig/rootlogon.C"
Error: G__unloadfile() Can not unload "/home/spataro/jan12/pandaroot/gconfig/rootlogon.C", file busy /tmp/SETUP_c7827247.C:20:
Note: File "/home/spataro/jan12/pandaroot/gconfig/rootlogon.C" already loaded
*** Interpreter error recovered ***
DONT MIND THE ERRORS HERE, ITS EXECUTED.
0309: ShowPackages
*** Package cache client:/home/spataro/.proof/packages ***
totale 4
drwxr-x--- 3 spataro spataro 4096 2011-10-31 13:34 libFairRoot
lrwxrwxrwx 1 spataro spataro 53 2012-04-12 09:20 libFairRoot.par -> /home/spataro/jan12/pandaroot/gconfig/libFairRoot.par

0309: Done
FairRunAna::RunOnProof(): The chain seems to have 1000 entries.
FairRunAna::RunOnProof(): There are 1 files in the chain.
You requested to analyze events from 0 to 0 that is 0 events!!!
It will be changed to analyze all events from 0 to the end of chain (1000), that is to analyze 1000 events.
FairRunAna::RunOnProof(): Starting inChain->Process("FairAnaSelector","",1000,0)

Info in <TProofLite::SetQueryRunning>: starting query: 1
Info in <TProofQueryResult::SetRunning>: nwrks: 8
-I- FairAnaSelector::Begin()
Looking up for exact location of files: OK (1 files)
Looking up for exact location of files: OK (1 files)
Info in <TPacketizerAdaptive::TPacketizerAdaptive>: Setting max number of workers per node to 8
Validating files: OK (1 files)
Info in <TPacketizerAdaptive::InitStats>: fraction of remote files 0.000000
-I- FairAnaSelector::Terminate(): fOutput->ls() still sending)
OBJ: TSelectorList TSelectorList Special TList used in the TSelector : 0
OBJ: TList MissingFiles Doubly linked list : 0
OBJ: TStatus PROOF_Status : 0 at: 0x6f72540
OBJ: TOutputListSelectorDataMap PROOF_TOutputListSelectorDataMap_object Converter from output list to TSelector data members : 0 at: 0x6f7e880
-I- FairAnaSelector::Terminate(): -------------
Lite-0: all output objects have been merged
FairRunAna::RunOnProof(): inChain->Process DONE
------------------------------------------------------------------------ --------
-------------- actual containers in runtime database -------------------------
-------------- runs, versions ------------------------------------------------
run id
container 1st-inp 2nd-inp output
-------------- input/output --------------------------------------------------
first Input:
OBJ: FairParRootFile /home/spataro/jan12/pandaroot/macro/global/allParams_20120412_92051.root : 0 at: 0x3239980
Root file I/O /home/spataro/jan12/pandaroot/macro/global/allParams_20120412_92051.root is open
detector I/Os: FairGenericParIo
second Input:
Ascii I/O /home/spataro/jan12/pandaroot/macro/params/all.par is open
detector I/Os: FairGenericParIo
output: none


Macro finished succesfully.
Output file is tracks_22Part_n1000.root
Parameter file is /home/spataro/jan12/pandaroot/macro/global/params_22Part_n1000_f0_sep.ro ot
Real time 6.85992 s, CPU time 1.3 s

spataro@briareos:~/jan12/pandaroot/macro/global$ root -l -q 'tracks_BARREL_1000sep.C("proof",10);'
root [0]
Processing tracks_BARREL_1000sep.C("proof",10);...
FairRootManager::OpenOutFile("tracks_22Part_n10000.root")
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [823.3 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [439.2 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [472.5 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [535.2 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [875.1 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [989.7 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [905.8 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [1039.3 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [921.4 MB/s]
[TFile::Cp] Total 2.30 MB |====================| 100.00 % [1063.4 MB/s]
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TGeoManager::Init>: Deleting previous geometry: FAIRGeom/FAIR geometry
Info in <TGeoManager::CloseGeometry>: Geometry loaded from file...
Info in <TGeoManager::SetTopVolume>: Top volume is cave. Master volume is cave
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: 125176 nodes/ 4346 volume UID's in FAIR geometry
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------

*** Break *** segmentation violation
Aborted (core dumped)



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0 0x00007fd220302e1e in __libc_waitpid (pid=<optimized out>, stat_loc=0x7fff49656b80, options=<optimized out>) at ../sysdeps/unix/sysv/linux/waitpid.c:32
#1 0x00007fd220297e0e in do_system (line=0x3967b40 "/home/spataro/jan12/fsbuild/etc/gdb-backtrace.sh 19800 1>&2") at ../sysdeps/posix/system.c:149
#2 0x00007fd2211adae7 in TUnixSystem::StackTrace() () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#3 0x00007fd2211b03b3 in TUnixSystem::DispatchSignals(ESignals) () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#4 <signal handler called>
#5 0x00007fd21534cc26 in operator<< <TGeoManager> (buf=..., obj=0x1f81900) at /home/spataro/jan12/fsbuild/include/root/TBuffer.h:380
#6 0x00007fd2152a7030 in FairBaseParSet::Streamer (this=0x1e73640/build/buildd/gdb-7.3/gdb/stack.c:293: internal-error: print_frame_args: Assertion `nsym != NULL' failed.
===========================================================


The lines below might hint at the cause of the crash.
If they do not help you then please submit a bug report at
http://root.cern.ch/bugs. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5 0x00007fd21534cc26 in operator<< <TGeoManager> (buf=..., obj=0x1f81900) at /home/spataro/jan12/fsbuild/include/root/TBuffer.h:380
#6 0x00007fd2152a7030 in FairBaseParSet::Streamer (this=0x1e73640/build/buildd/gdb-7.3/gdb/stack.c:293: internal-error: print_frame_args: Assertion `nsym != NULL' failed.
===========================================================


Root > Function tracks_BARREL_1000sep() busy flag cleared
Fatal in <TFileMerger::RecursiveRemove>: Output file of the TFile Merger (targetting /home/spataro/jan12/pandaroot/macro/global/allParams_20120412_94806.root ) has been deleted (likely due to a TTree larger than 100Gb)
aborting
#0 0x00007fd220302e1e in __libc_waitpid (pid=<optimized out>, stat_loc=0x7fff49679720, options=<optimized out>) at ../sysdeps/unix/sysv/linux/waitpid.c:32
#1 0x00007fd220297e0e in do_system (line=0x1bd4e60 "/home/spataro/jan12/fsbuild/etc/gdb-backtrace.sh 19800 1>&2") at ../sysdeps/posix/system.c:149
#2 0x00007fd2211adae7 in TUnixSystem::StackTrace() () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#3 0x00007fd2210f79a2 in DefaultErrorHandler(int, bool, char const*, char const*) () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#4 0x00007fd2210f7eae in ErrorHandler () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#5 0x00007fd221107394 in TObject::Fatal(char const*, char const*, ...) const () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#6 0x00007fd22116231b in THashList::RecursiveRemove(TObject*) () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#7 0x00007fd221108198 in TObject::~TObject() () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#8 0x00007fd21dfebf79 in TFile::~TFile() () from /home/spataro/jan12/fsbuild/lib/root/libRIO.so
#9 0x00007fd221161259 in TCollection::GarbageCollect(TObject*) () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#10 0x00007fd221164be5 in TList::Delete(char const*) () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#11 0x00007fd22111d087 in TROOT::~TROOT() () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#12 0x00007fd22028fc3d in __cxa_finalize (d=0x7fd22188ac00) at cxa_finalize.c:56
#13 0x00007fd2210dc126 in __do_global_dtors_aux () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#14 0x00007fff4967bcf0 in ?? ()
#15 0x00007fff4967c0d0 in ?? ()
#16 0x00007fd221517871 in _fini () from /home/spataro/jan12/fsbuild/lib/root/libCore.so.5.32
#17 0x00007fff4967c0d0 in ?? ()
#18 0x00007fd22191e135 in ?? () from /lib64/ld-linux-x86-64.so.2