.. _alignment-tutorial: Alignment ========= This chapter describes the manual alignment procedure based on the unbiased residuals. For exact files reference we took 2022-mu data here. Refer to :ref:`alignment` in "Advanced Topics" for conventions on coordinate systems, angles and units. By *alignment* we imply a procedure of finding the positions and orientation of detector entities that would provide more consistent track fitting picture in terms if certain track hypothesis. That means, if things done right we shall end up with geometrical picture more precise than one provided by survey. The goal is to minimize discrepancy that one observed in residuals plots as a shift of the mean value and standard deviation. Default tracking pipeline provides 1D residuals distribution and there is a dedicated handler providing dump of residual values. Alignment 2022: running pipeline -------------------------------- In this subsection we describe a draft recipe for the alignment procedure for 2022-mu data. .. warning:: The procedure is not yet finally shaped, so some of NA64sw internals that normally should be hidden by runtime path(s), default settings and modules are exposed. During the alignment we will modify some parameters of the pipeline config, so let's start by making local copy of the assets on lxplus: .. code-block:: shell $ mkdir na64-alignment-2022 $ cd na64-alignment-2022 $ source /afs/cern.ch/work/r/rdusaev/public/na64/sw/LCG_101/x86_64-centos7-gcc11-opt/this-env.sh $ cp $NA64SW_PREFIX/share/na64sw/run/dev/2022-alignment.yaml . $ cp $NA64SW_PREFIX/var/src/na64sw/extensions/handlers-genfit/presets/magnets-2022.yaml . We've copied calibration settings at the last line as we would like to override it with our custom data (namely, placements). Normally it is taken from system dirs, but we're going to override it with ``-c,--calibrations`` option. One can run the pipeline app on the config's copy, but since GenFit2 tracking handlers used by this config are still at the experimental stage, one have to explicitly load GenFit2 module with ``-m genfit-handlers`` argument provided to pipeline app invocation. Since magnetic field description is not yet handled by calibration indeces we have to also explicitly specify it with parameter ``-EROOTSetupGeometry.magnetsCfg=...``. So the full command to run the reconstruction: .. code-block:: shell $ na64sw-pipe -m genfit-handlers \ -EROOTSetupGeometry.magnetsCfg=./magnets-2022.yaml \ -r ./2022-alignment.yaml /eos/experiment/na64/data/cdr/cdr01001-006030.dat \ -N 5000 This call will run pipeline application performing reconstruction of 5000 events (``-N 5000``) from file ``/eos/experiment/na64/data/cdr/cdr01001-006030.dat`` as described in ``./2022-alignment.yaml`` with GenFit2 extensions available and magnetic field configured as in ``./magnets-2022.yaml``. Alignment 2022: input and output -------------------------------- There are two ouput artifacts generated by this pipeline that we're interested in: the ``processed.root`` file that keeps various histograms and ``alignment.dat`` file which contains dump of the residuals distribution. First, have a look at the ``processed.root`` file. There would be 1D residuals plots generated for each detector participating in tracking (navigate to, for instance, ``MM/6/alignment/Y/residuals-1D-u-straight``. Here ``...u-straight`` corresponds to *unbiased* residuals and ``..._b-straight`` is for *biased* residuals, of the straight tracks segments). .. figure:: https://cernbox.cern.ch/index.php/s/TNGes1TQkZz9zQi/download :alt: (example picture of MM's residuals plot in TBrowser here) Central peak apparently originates from trivial tracks that pass through this plane and just a single one else, making its standard deviation to be unphysically small. During the alignment the goal is to make this distributions centered and decrease standard deviation to best possible value (typically corresponding to detector's resolution). On the picture attached, one can see three major peaks that may be caused by misalignment, hardware or reconstruction issues. It is hard to tell more, judging just by 1D picture. There is also a general track fitting plots in this ROOT file available in the ``alignment`` dir, like a p-value distribution (yet it starts only make sense if you have more than three stations enabled in track reconstruction). More details may be provided by other output -- the ``alignment.dat`` file. It contains dump of the residuals for every detector per each reconstructed track. It can be processed by the Python script to derive corrections for the *placements file* (thus, to align detectors). Alignment 2022: reading the data -------------------------------- Jupyter notebook would be apparently the most conveninet tool to handle the ``alignment.dat`` output and evaluate corrections. I've found it convenient to run it locally in the following way: .. code-block:: shell $ mkdir na64alignment/ $ cd na64alignment $ python3 -m venv venv $ source venv/bin/activate $ pip install --upgrade pip $ pip install jupyter-notebook pandas seaborn Copy ``plots.py``, ``sdcDoc.py`` and ``unbiased.py`` modules from ``na64sw/utils/alignment/`` dir of your (remote) repository to this newly created ``na64alignment/`` dir. There is also a `template notebook file`_ here that you can download and put to this dir. .. _template notebook file: https://cernbox.cern.ch/index.php/s/wj4JLv2TiyP2wPa Run the notebook with ``$ jupyter-notebook`` and open the ``alignment.ipynb``. I've tried to write annotation for cells. In principle, the procedure should be pretty straightforward, as described -- put the ``placements.txt`` and ``alignment.dat`` files at the current directory, run the cells and replace ``placements.txt`` on remote with ``placements.updated.txt``. BUT. .. figure:: https://cernbox.cern.ch/index.php/s/g5tDmsEerhMyQm7/download :alt: (Example of 2D profile plot picture here) Example of 2D profile plot (v/du). A clear shift is seen here at ~8mm indicating that assumed plane position is shifted wrt one in current placements. Regression fit lied at around 5mm. But there is a issue with this procedure the linear regression is not always adequate in the natural sense. The problem is that the noise around the "bars" is not actually uniform and its contribution distorts LMS fitting. We are trying to mitigate it wih *a posteriori* with some clustering techniques, so far with no much luck. Currently, the only solution we've got is to adjust *du-cut* of the plot (marked with a corresponding comment in the notebook) to some reasonable values and apply procedure iteratively. Procedure outline ----------------- Currently, the proposed plan is to perform a local alignment manually, by steps: #. First, choose a couple of stations in each region with straight tracks. They will be *pivots* in this region, meaning that their relative positions will define local coordinate system and we won't touch them. It is better to choose first and last station in the region. #. Then select a single station in the region that has to be aligned with respect to this pair and perform few iterations of unbiased alignment **only** on this station (keep pivots untouched). #. Select another station and do the alignment of this station only, this time keeping one aligned at previous step(s) together with pivots untouched. If done right, this procedure shall end up with stations decently aligned with respect to each other for the straight track regions. There are two parameters to control: detectors included in tracking (as we want to control which detectors will be on or off) and overriden placements file. Overriding placements ~~~~~~~~~~~~~~~~~~~~~ NA64sw calibrations are initially configured by ``calibrations.yaml``. By default applications expect it to be by system path, but we can override it for pipeline app with ``-c,--calibrations`` option. Within this file we can re-define locations used to discover calibration files. Let's make a local copy of calibration files and config (assuming we're in ``na64-alignment-2022/`` dir created at the beginning of this chapter): .. code-block:: shell $ cp -r $NA64SW_PREFIX/var/src/na64sw/presets/calibrations my-calib $ cp $NA64SW_PREFIX/var/src/na64sw/presets/calibrations.yaml . This config is bulky as it sets up the whole thing, but placements are provided by SDC calibrations auto-discovery. Find the ``SDC.byRunNo.paths`` parameter (should be near line 194) and re-define it with, say: .. code-block:: yaml paths: "my-calib/:${P348_CONDDB_DIR}/calib/" instead of ``paths: "${NA64SW_PREFIX}/share/na64sw/calibrations/..."`` or whatever. Rename original placements from ``my-calib/override/2022/placements.txt.bck`` to ``my-calib/override/2022/placements.txt``: .. code-block:: shell $ mv my-calib/override/2022/placements.txt.bck my-calib/override/2022/placements.txt and run the pipeline app this time with ``-c calibrations.yaml``. If done right it must be visible in the log like: .. code-block:: shell utils.genfitGeoCache INFO : Using 22 placement entries from my-calib/placements.txt (do not mind if it appeares few times). .. warning:: Beware of having few copies of ``placements.txt`` for same period in this dir as they discovered automatically and will be applied all at once. This is kind of intentional behaviour. This way one may substtute ``placements.txt`` file with new (updated) copy. Controlling detectors ~~~~~~~~~~~~~~~~~~~~~ Detectors enabled in tracking are specified in the pipeline config file -- ``alignment-2022.yaml`` that we copied previously. At the beginning of this file a handler named ``CreateScore`` has ``applyTo`` parameter which essentially controls for which detectors to produce the scores for tracking. Its syntax is just a logical expressions supporting basic comparison (``>``, ``<``, ``==``, ``!=``) and logic (``||``, ``&&``) operators, very similar to C++ (parenthesis supported). The operands may be numbers or, in certain cases aliases, like ``kin``, ``projection``, ``X``, ``Y``, ``MM``, ``GM``, etc. Examples: .. code-block:: yaml applyTo: kin == MM # scores created for all the MMs applyTo: kin == MM && number != 3 # for all the MMs except #3 applyTo: kin == GM || (number > 4 && kin == MM) # all GMs, MMs starting from 4th This thing is perculiar, but it was done this way for the sake of computational efficiency. Tip: event display ~~~~~~~~~~~~~~~~~~ You may run slightly customized GenFit2 event display application to fet an overview of the detectors and their hits. Unfortunately, it seems impossible to run it over SSH as it requires OpenGL. However in case of local build or ``/afs`` mount on local machine it is possible. To make it running: #. Set/uncomment ``showOnGenFitEventDisplay`` option of ``GenFit2_FitTrack`` handler to ``true`` in the pipeline config. #. Provide ``-EGenFit2EvDisplay.enable=yes`` option to command-line invocation of ``na64sw-pipe``. You probably would like to set the number of events to smaller amount as well with ``-N`` (otherwise, event display app will keep everything in RAM). .. figure:: https://cernbox.cern.ch/index.php/s/LDKZsYYrJVMnPJp/download :alt: (example of event display view here) A front view on one of the first versions of 2022-mu placements. One can identify couple of gems below the beam line. ROOT TEve's event display is quite tricky to manipulate though, but it can be useful to observe sime obvious issues.