Home » PANDA » PandaRoot » Tracking » Benchmark Tables and plots for tracking performance
| Re: Benchmark Tables and plots for tracking performance [message #11124 is a reply to message #11107] |
Wed, 27 October 2010 09:09   |
StefanoSpataro
Messages: 2736
Registered: June 2005
Location: Torino
|
first-grade participant |
From: *vpn.unito.it
|
|
With some delay here you are my comments to Gianluigi proposal.
| Quote: |
I would generate tracks with 5 degrees< Theta < 120 degrees, in the full Fi range, with the Box Generator.
The vertex could be randomly distributed at +/- 0.4 cm from (0,0,0).
|
I would prefer to separate effects coming from displaced vertex. I think it would be better to run IP (0,0,0), and only in a second time with some vertex smearing.
| Quote: |
The benchmark procedure would consist of :
a) Generation of events with 1 track, 5 tracks, 10 tracks at different total momenta with the Box Generator to check the performance at different phase space regions.
|
Fine for me.
| Quote: |
For instance :
at 0.1 GeV/c 1 track, 5 tracks, 10 tracks
at 0.3 GeV/c 1 track, 5 tracks, 10 tracks
at 1. GeV/c 1 track, 5 tracks, 10 tracks
at 5. GeV/c 1 track, 5 tracks
at 10. GeV/c 1 track, 2 tracks
|
Not fine for me. The low momentum region has performances more sensitive to momentum, and it needs a finer check. I would propose a flat distribution from 0.1 to 2 GeV, and then two more points 5 - 10 GeV, or 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.5 2 5 10.
| Quote: |
b) Exclusion from analysis of montecarlo tracks producing
less than a predefined number (for instance : 3) hits in the Panda central detector. Also exclusion of those MC tracks not originating from the interaction vertex.
|
Maybe you mean exclusion of reconstructed tracks with less than the number. Or you want to separate acceptance from reco efficiency? I would consider only primary particles.
| Quote: |
c) Assignment of each track found by the global Pattern Recognition to one (if possible) MC track.
(Naturally and unfortunately there is no unique way of assigning a track found by PR to the corresponding MC track. See below the method I suggest to use).
d) COMPUTATION OF A TABLE WITH THE % OF TRUE HITS FOUND,
THE NUMBER OF HITS NOT BELONGING (SPURIOUS); IF A TRACK IS NOT ASSIGNED TO ANY MC TRACK, increase the counter of spurious tracks. IF A MC TRACK HAS NO PR TRACK ASSIGNED, increase the counter of unmatched tracks.
This numbers are the Benchmark Table.
|
I have not well understood your method, could you please do an example? I don't think there could be reco tracks without mc assignment (apart matching a series of mvd noise hits...). And you have not said when you assign a mc id to a particle (in each case, the most common mc id? only with at least 60% of mc id? what?) [Ok, I have seen there are some explanations at the end]
| Quote: |
2- Benchmark Plots, concerning momentum resolutions and physics
channels performance.
For these plots PR + momentum fitting (Kalman with Genfit) is needed.
|
We need also momentum after prefit, without kalman. This tells us what the kalman is really doing.
| Quote: |
a) Plot of the residual of total momentum for all momenta and track topologies generated as said before.
In other words : plot of
[(Total Reconstructed Momentum) - (MC truth Momentum)]/
(Error on Total Reconstructed Momentum)
|
This tells you only the quality of your fit, it should centred at zero with a "1" sigma. You need also as a function of theta?
| Quote: |
b) Plot of the residual of Pper (Momentum in XY plane) for each momentum and all topologies.
|
and theta.
I think we need momentum residual and mom resolution as a function of momentum and theta, theta residual and resolution, and efficiency also for prefitting and fitting (how to define this efficiency? within 5 sigma?). For each particle multiplicity.
| Quote: |
c) Plot of some typical physical EXCLUSIVE channel, with all charged particles. An examples could be :
|
I think this comes later. First we need to validate reconstruction apart from physics channels. We will need them in a second time.
| Quote: |
a) first calculate a 2-dimensional Matrix ; the number of rows are equal to the # of tracks found by PR, the # of columns is equal to the # of MC tracks having at least 1 hit in any of the tracks found by PR;
each cell of this Matrix is filled with the corresponding number of hits.
For instance, suppose that the track # 2 found by PR has 7 hits
belonging to MC track # 5, that the cell (2,5) is filled with 7.
Do the filling looping over all tracks found by PR.
b) Find the cell in the Matrix with the largest number. Assign the corresponding track found to PR to the MC track.
For instance, suppose the cell (5 ,  contains 25, and that is the maximum value in the whole Matrix. The the track found # 5 is assigned to the MC track #8.
Erase the row and column just matched ( row 5, column 8 in the example)
c) Consider the remaining Matrix; iterate the procedure in b) and assign another track found by PR to a MC track. Iterate untill ALL MC TRACKS or ALL TRACK FOUND BY PR have been exhausted.
|
Could you please provide a standard macro with such kind of code, that starting from PndMCTrack TCA and PndTrackCand TCA is able to do this job and fill the corresponding tables/plots? Probably you have it already, done for STT. I need some additional time to think about this methid, but I think it could be a good starting method.
It should be the same for all the tracking codes, then we don't have to repeat the same job (I mean, writing the code) many times (PndRecoQATask ?). You have clearer and smarter idea on the subject than me. In this case, the tracking developer should only produce the statistics, filling the table and the plots, and cross checking what is going wrong, and we introduce less sistematics from "buggy code" ... 
Maybe, writing a global reco qa task could be even better, so that we can use it for dashboard; I like this idea.
This discussion is open to everybody, and other comments are welcome!
|
|
|
|
Current Time: Fri Apr 26 10:59:45 CEST 2024
Total time taken to generate the page: 0.01058 seconds
|
GSI Forum
Search
Help
Members
Login
Home
