GSI Helmholtzzentrum für Schwerionenforschung
[FIXED] Problem with mass constraint fit for two gamma
Hi I found a strange behaviour with mass constrain fit. For charged combination (here is e.g. D0->K-pi+), chi2 distribution of mass constrain fit are similar and seems to be OK for both full and fast simulation. An issue about mass constraint fit has been fixed in few weeks ago. (see message #16518 : Problems with PndKinFitter::AddMassConstraint). However one more problem is still remaining. For neural combination (here is e.g. D0->gamma gamma), chi2 distribution and also probability distribution of mass constrain fit are too different in the case of full and fast simulation at scrut14. I have correcly delta function for invariant mass for gamma gamma combination after mass constraint fit. Only problem is the two low chi2 distribution(distributed in below 0.1) in the full simulation. full_gg_mass_fit_results_to_forum_charged.eps show the chi2 distribution of the mass fit for D0->K-pi+ in two cases(fast and full). full_gg_mass_fit_results_to_forum_neutral.eps show the chi2 distribution of the mass fit for D0->gamma gamma. Below is the accessor what I have commonly used for full and fast with scrut14 version. //Pnd Kin Mass Fitter PndKinFitter mfitter(dp[j]); // instantiate the kin fitter in mfitter.AddMassConstraint(D0Mass); // set 4 constraint mfitter.Fit(); // do fit double chi2_m = mfitter.GetChi2(); // get chi2 of fit double prob_m = mfitter.GetProb(); // access probability of fit RhoCandidate *dpfit_mass = dp[j]->GetFit(); // get fitted D0 I don't understand why the chi2 distributions are so different for the neutral combination in both case(full and fast)? I assume that some factors are missing or NDF is handled wrongly for the neutral combination case in the full simulation . Best wishes, Donghee
Re: Problem with mass constraint fit for two gamma
Hi Donghee and all, this situation is the same in reconstructing pi0->gamma gamma, as I reported before. In the fast sim, the invariant mass spectrum of the combination of two photons has a "nice" distribution peaked at pi0 mass, so it's understandable that the mass constraint fit can work as expected. In the full sim, the invariant mass distribution is filled with a significant "background" which may course problems of convergence in mass constraint fit, e.g. too small chi2, prob squeezed to 1. Since the PndKinFitter:AddMassConstraint() works fine for charged particles as well as the neutral in the fast sim, I'd like to guess the tool itself has no problem but to understand why the two-photon invariant mass distribution has so much "background" in the full sim. Is there any possible incorrect combinatorics, or where're the extra photons coming from? Donghee, could you also show the gg invarit mass spectrum in your case? Best regards, Lu
I repeat for the nth time: the EMC covariance matrix in full simulation was never filled properly, and there is currently nobody from EMC groups working to have a proper implementation. In fast everything is fine since the error matrix is what you impose in the simulation. Lu, your plot is wrong, if you use a mass constraint fit then you forse your background to stay inside your peak. The fit is not done to remove background, but to select your events once you have already removed the background.
Hi Stefano, I will be a volunteer to help this issue. (will try, but cannot guarantee about completeness) First of all, I have to know what is the difference for the covariance matrix in EMC part, and better to compare it between fast and full. Where is the EMC covariance matrix in the full simulation? Best wishes, Donghee
Hi Stefano, In input folder, there are couple of emc_matrix as a root format, but I cannot read it. Who produce them? Is this file some kind of mapping according to momentum, theta, and phi? Best wishes, Donghee
Hi Lu, Left is for the D0 (gamma gamma) and right is after mass constraint fit. As you mentioned, chi2 is squeezed to 0 also in my case. But I guess that the background issue is not connected with this problem of mass fitter. We need to solve matrix problem of EMC in the case of full simulation, then chi2 distribution should be reasonable. Best wishes, donghee
Hi Donghee, The EMC error matrix is accessed via class /emc/EmcReco/PndEmcErrorMatrix.cxx and it is used in PndPidCorrelator to actually set error matrix for neutral candidates. The parameters used in this parametrization are stored in /input/emc_error_matrix_default.root and /input/emc_error_matrix_1.root Error matrix is parametrized on energy and position of the cluster with independent parametrizations for Barrel Forward and Backward endcaps. I have attached the presentation from Mark Pelizaeus (I hope he does not mind) who did it in old framework and I adapted it for pandaroot. I remember he had another presentation on the subject but I cannot find it now. emc_error_matrix_default.root contain parameters obtained by Marc in old framework and in emc_error_matrix_1.root there are parameters which I calculated for one emc geometry ("emc_module12.dat","emc_module3new.root","emc_module4_StraightGeo24.4.root ","emc_module5_fsc.root"), but it is not exactly the one used by default now. For forward endcap it was modified to "emc_module3_2012_new.root". There are several macros related to emc error matrix in /macro/emc/dedicated/ error_matrix_data_production.C - produces photons in the whole emc range to calculate error matrices error_matrix_fit.C - to actually calculate error matrices, fit them, and store parameters in PndEmcErrorMatrixParObject containers. fill_error_matrix_param.C - to fill PndEmcErrorMatrixParObject container with given parameters. I just found that in PndEmcErrorMatrix.cxx the emc_error_matrix_default.root was used and not emc_error_matrix_1.root for default geometry and I will correct it now (within one hour) and you can try if this can solve the problem. Otherwise it makes sense to start with some QA macros to test pull distribution of pi0 before and after 4C-fit as suggested in Mark's presentation and if there is actually a problem with error matrices, to produce photons, re-calculate error matrices, parametrize them and test again. Dima
Hi Stefano, Here is the chi2 distribution for D0->gamma gamma mass constraint fit. (detail plots are in the file at first posting of this title) Black histrogram is the all candidates and dashed blue is for the MC truth matched one. I assume that the chi2 values are normalized some factor. I will discuss with Dima. Best wishes, Donghee
I just updated the PndEmcErrorMatrix.cxx and now the emc_error_matrix_1.root error matrix parametrization is used for default emc geometry instead of emc_error_matrix_default.root. There is a chance that it can improve the results.
Hi Dima, Thank you for your kind explanation. I think that you have already made all relevant study for error matrices. It means that in fast simulation the error matrix "emc_error_matrix_1.root" has been accessed correclty but in full simulation old parameterization emc_error_matrix_default.root was used up to now. Did you make parametrizations for Barrel, Forward endcap, Backward endcap, forward EMC, all 4 parts separately? what is the difference between "emc_module3new.root" and "emc_module3_2012_new.root". Your parametrization is based on the former, but the default mapper in PANDAroot (version=1) is the latter. Do we need a re-calculation of error matrix using modul "emc_module3_2012_new.root"? If you think that we need some huge photon production in each kinematic binning and more study for error matrices, please let me know I am ready to do that. Best wishes, Donghee
Hi Dima, I try again with modified PndEmcErrorMatrix.cxx and the emc_error_matrix_1.root, but chi2 distribution is not improved. It is still squeezed as before. I am wondering why doens't change anything? Either We need to recalculation for error matrice or have to find some bugs during the filling the error of matrices in the code. Best wishes, Donghee
Hi Donghee, I try to repeat again: the current EMC error matrices were not calculated with the current EMC geometry and reconstruction code, they should be recalculated.
Hi, It's pity that it does not help, i.e. usage of emc_error_matrix_1.root instead of emc_error_matrix_default.root. Comment to Stefano, that emc_error_matrix_1.root was calculated for geometry which is different from current only for forward endcap and modification in reconstruction code should not be (?) relevant for this case. Comment to Donghee > It means that in fast simulation the error matrix "emc_error_matrix_1.root" has been accessed correclty but in full simulation old parameterization emc_error_matrix_default.root was used up to now. No, in fast simuilation the error matrices were calculated/parametrized independently. > Did you make parametrizations for Barrel, Forward endcap, Backward endcap, forward EMC, all 4 parts separately? There is a separate parametrization for each EMC part but I calculated them in one macro. But the conclusion is that error matrices should be recalculated at least for modified forward endcap, but parametrization/fit should be checked anyway, since I do not suppose that only forward endcap introduces problems. So it makes sense to start with some QA macro to look for pull distribution for pi0 before/after 4C-fit and only later to recalculate error matrices. Dima
Hi Stefano, Dima suggested simply solution as like this, but it doesn't make sense...it is pity. I will discuss with him and try to produce correct matrix with current geometry. Best wishes,
Hi Dima, If I remember correctly, last year there was the open question about non-uniformity light collection and electronic noise, and I believe the error matrices were calculated before we were settings the proper flags. In this sense I believe that this has screwed a bit the situation.
Hi Dima, I am starting to look the re-etsimation of the error matrices for EMC. Clearly We need some modifications of QA macros. And I found an unclear thing for the photon production, why all other detectors are turned off when the photons are produced? In order to take into account material effect, MVD,STT,Drc and so on every detector system have to be put into the simulation. Best wishes, Donghee
Hi Donghee, It's actually a question I do not have an answer, to include all the other subdetectors or not when calculationg error matrices. Other subdetectors can cause a preshower with some probablility and photons reconstructed after preshower will have worse resolution, even if they are reconstructed at all. And I do not know if to include them in error matrix calculation or not. But I do not have strong arguments against it, so you can use the whole PANDA for error matrix calculation if you want. Dima
Hi Dima, I have some technical question for ths simulation. We need to know the energy of emc cluster in this task. There are two quantity, one is the raw energy and the other one is corrected energy by GetEnergyCorrected() from clustering. In my understanding, corrected energy is the energy deposit in the cluster after calibration. We are using normally corrected energy in the analysis level, so I assume that we have to use corrected one for the error estimation too. Otherwise, error matrix should be overestimated without correction(calibration). Best wishes, Donghee
Hi Donghee, Actually I do not see a problem using raw energy for error matrix assuming the same energy is used for parametrization and calculation of error matrices. I.e. error matrix is calculated for the proper raw energy (in PndPidCorrelator) but may be the error itself will be expresed in non-correcterd energy units but I think it's a second order effect here. On the other hand using corrected energy will complicate the code. The method GetEnergyCorrected() of the PndEmcCluster is obsolete and in principle should be removed. You obtaine corrected energy with PndEmcClusterCalibrator class, as for example (fEmcCalibrator->Energy(cluster)) and using correcte energy will create dependence of PndEmcErrorMatrix class on PndEmcClusterCalibrator, which is not a problem if it's necessary, but I am not sure it is. Dima
Before starting to calculate matrices, if I were in you I would check first the actual code. First, is the information stored in the neutral candidate correct? Or is maybe the error matrix somehow bugged? Maybe something in the initialization went wrong? When I launch a pid macro i can see the following error: -I- FairRunTimeDB::InitContainer() PndEmcErrorMatrixPar [ERROR ] init() PndEmcErrorMatrixPar not initialized I could suspect the par is not properly loaded. Then, if I launch the photon simulation I can see fCov[15] is not set (it is at zero) while fErrP7[28] has strange numbers, also negative (-30?). If I do GetErrorP7() I obtain something at 10^6! Are we sure the correct data members are properly filled? Which variables are needed really for the kinematic fitters? Maybe we are just filling the wrong thing, or maybe we are filling it in the wrong way (i.e. position inverted with momentum, mass at the beginning and not at the end, the usual stuff). Could you please someone do such checks?
Hi Stefano and Dima, I am now looking your posting for the neutral candidate in pid correlator. Launching the PID macro, Error message doesn't have serious meaning because we didn't store the emc parameter (PndEmcErrorMatrixPar) into the data base. Quote: // Get Emc error matrix parameter container fEmcErrorMatrixPar = (PndEmcErrorMatrixPar*) db->getContainer("PndEmcErrorMatrixPar"); We can ignore this alarm, or we can make inactive thie matrix parameter container before initialization, because we will access error matrix from the input reading file later on. Quote: void PndPidCorrelator::SetParContainers() { ... // Get Emc error matrix parameter container fEmcErrorMatrixPar = (PndEmcErrorMatrixPar*) db->getContainer("PndEmcErrorMatrixPar"); ... } I find a potential problem for miss matching of neutral candidate. When we construct a neutral candidate using EMC, we build candidates from "bump" information based, not EMC "cluster". But error calculation is previously made by EMC cluster. I don't know exactly which one has more fundamental information and is placed more high rank. Best wishes, Donghee
Hi Donghee, there is no problem using cluster for error calculation and applying them to bumps since PndEmcBump is in principle a PndEmcCluser except for the cases when PndEmcCluser is produced from pi0 and therefore cluster has two local energy maxima and is divided into two bumps, otherwise EmcBump is a copy of EmcCluster. Dima
Hi Dima and Stefano, Error matrix in the contructuion of nuetral candidate has a form with 7X7 matrices in total 3 categories. I have seen/learn follow setting for neutral cand. with PndPidCandidate. position error set to be 0. position-momentum covariance set to be also 0. momentum(px,py,pz,E) error has been evaluated by module and cluster. I am wondering the term in position-momentum covariance must be 0 for neutral case? or I am now misunderstanding for this concept. Best wishes, Donghee
In theory the kinematic fitter should not use position neither pos errors.
Dear Donghee, Could you please update the Emc folder of the trunk and try again? I think I found and fixed the bug.
Dear Stefano, Thanks a lot. I submit a job with this new EMC reconstruction part. Probably, I focused on the wrong direction before getting your e-mail. You can see what I was suspicious about and discussion with Dima at today morning. See you soon, ------------------------------------------------------------ Hi Dima, I am produing emc data(photon) to get error matrices according current geometries. Everythings is more or less smooth. I found that the position resolution of backward endcap are really bad, if I compare with forward end cap. Did you see also same behaviour? I assume that our main problem is the scaleing factor in the PndEmcErrorMatrix.cxx We are calculating the position error at z or R=100cm when we estimate errors and save it from the analysing photon data. Then in PndEmcErrorMatrix we make a rescale with 54/100*error, which 54 is the radius of barrel for example and 100 is previouly given in the error matrix data that have to be subtracted during the access this information at analysis (fit) level. But I think that this is something wrong. we don't need any scale factor. All error are estimated by differences at certain position. It has been already taken into account during transformation in which is TMatrix trans(4,3). I am trying to do with following way. In PndEmcErrorMatrix, I change from pos1Err *=scaleFactor pos2Err *=scaleFactor to pos1Err *=1/100 pos2Err *=1/100 let see what should happen. Donghee ------------------------------------------------------------
Hi, the mine was a bug, but the matrices need to be recalculated anyway. I hope that with my correction the chi2 will assume something closer to what we should expect, but for the correct analysis we need better matrices for sure. No idea about the scaling factor you are speaking about. I can try to take a look tomorrow.
Dear Stefano and Dima, I am very happy to inform you that the problem is finally solved. Test two plot show the chi2 distribution for D0(gammagamma) mass constrain fit before and after fixed error matrix in PndEmcErrorMatrix. One need to re-run pid correlator to have correct error matrix during the analysis. In addition, I will update soon more precise estimation of error matrices according current EMC geometry & will report about it in our PANDAroot meeting. Thanks again to Stefano for your valuable effort and great help.
Very good, I will copy this modification also to the scrut14. Do you want to report then next wednesday?
Dear Stefano, I am pleasure to do that. I will present about update of EMC error matrix with new geometry including some diagnostics(mass constraint fit) from pi0 and D0 data". Best wishes, Donghee
Hi Stefano, why my plot is wrong?? It's the invariant mass distribution of two-photon BEFORE any fittings. Where's the problem in this plot? Sorry for delayed response. Best, Lu
It seemed like an old plot with mass after and before the fit. If this is for two different samples, then maybe it is ok. I don't remember which was the discussion exactly 20 days ago. I would suggest to close this topic and to open some new one, since here there are two many ongoing discussions.