ML searches with LT on 06 sims

 (C. Umiltà)
Updated on 2020-08-28

Update on Sep 28, 2020: The bias issue whown in this posting has been resolved. updated results can be found here .

Comparison with previous study

In a previous study I run ML searches on DC02 sims with and without using the lensing template. Quoting from my old posting "(...) the mean of the simulations does not correspond to the exvals of the model: this could be the reason of the observed bias in the recovered r. For this reason in all simulations I subtract to the simulations the mean bias, i.e., the difference between the expectation value and the mean of the sims. The bias in r is largely removed by subtracting the mean of the sims."

The old script is the same I used with DC02 sims. The new script is adapted from the script 'make_s4_simset_XXp00_driver.m'. At the beginning I though differences came from the change in script, but it does not seem to be the case. You can only see small differences for r=0.003.

Technical note: We only use 100 sims for both DC02 and DC06. The lensing template used is the 'ideal' one. In the figure, I use the same binning for the 'broad' distributions, but I do not constrain the binning for the thin orange ones.

I run the same ML searches on DC 06. I noticed a few things:

Figure 1:
ML searches for DC02 and DC06. Orange histograms use LT in the ML search. Purple histograms come from final files that conain the LT but exclude it using 'expt_select' function. Blue histograms come from final files that have no LT.

The table below summarizes mean and 1-\(\sigma \) values for the histograms (reporting values only for the new script). Values are multiplied by 1000 for readability.

1733 (DC02 no LT) 4331 (DC02 with LT, LT ignored) 4326 (DC06 with LT, LT ignored) 4331 (DC02 with LT) 4326 (DC06 with LT)
standard, r=0 \(-0.9 \pm 2.7\) \(-0.9 \pm 2.8\) \(-0.3 \pm 3.2\) \(-1.49 \pm 0.28\) \(-0.850 \pm 0.070\)
with mean subtraction, r=0 \(0.0 \pm 3.0 \) \(0.2 \pm 2.8\) \(0.5 \pm 3.3\) \(-0.03 \pm 0.30\) \(-0.017 \pm 0.075\)
standard, r=0.003 \(1.9 \pm 2.2\) \(1.8 \pm 2.2\) \(3.2 \pm 3.3\) \(1.29 \pm 0.44\) \(1.52 \pm 0.25\)
with mean subtraction, r=0.003 \( 2.8 \pm 2.2\) \( 2.9 \pm 2.2\) \( 4.1 \pm 3.4\) \( 2.99 \pm 0.47\) \( 3.03 \pm 0.33\)

Overall we see that the bias is still present in DC06, but it is slightly reduced. Using the ideal LT the distributions are also slightly narrower in DC06 than DC02. In contrast, if we run the ML search without the LT, we see a slightly more biased and broader distribution, with or without mean subtraction.


In the figure below, I reproduce Clem's post figure on bpwf and add the variables that I use in the ML script. I compare here dc06 and dc02. This is to check if we see some difference. The 'Spec' click compares simulation mean and expectation values for both Clem's script in blue/cyan and mine in red/magenta. The 'Ratio' click shows ratios between the mean of sims and expvals. The last click shows the factor I actually subtract to each sim that ends up reducing the bias (mean of sims - expvals). In the 'Ratio' and 'Mean subtraction factor' clicks the ylims are fixed and equal for all plots on the grid.

Figure 2:
Mean of sims and expvals for auto- and cross- spectra..

These figures do not show any particular difference in the data I used wrt the data Clem used. So the bias has to arise from somewhere else. In an email exchange, Clem suggested that " [...] it is something to do with the forcing of the mean of the LT spectra values to the exp vals - where the corresponding non-LT spectra are allowed to have their natural deviations from the exp vals". And a possible soluton would be to force the mean for all BB spectra to be exactly as the expvals. Doing this seems in fact to remove the bias, as shown in Figure 3.

Figure 3:
ML searches when forcing mean of sims to expvals.