Achieved performance roundup

(C. Bischoff)
Updated minor edits and clarifications

1. Measuring $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$

The first goal is to describe the sensitivity of as many recent CMB polarization results as possible in terms of $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ (which can also be a function of $$\ell$$). These estimates should be drawn from bandpower error bars, so they account for all sources of inefficiency, including variations in per-detector or instantaneous sensitivity, cuts, non-uniform map weight, masking / apodization, non-optimal power spectrum estimators, etc.

I will make use of a Knox formula relationship between $$\mathcal{N}_\ell$$ and bandpower error bars

$\begin{equation} \sigma \left( \mathcal{C}_{\ell,\mathrm{obs}} \right) = \sqrt{ \frac{2}{k} } \left( \mathcal{C}_{\ell,\mathrm{theory}} + \mathcal{N}_\ell \right) , \label{eq:sigma} \end{equation}$

where $$k$$ is the bandpower degrees-of-freedom (for a bandpower than runs from $$\ell_0$$ to $$\ell_1$$), which can be related to $$f_\mathrm{sky}$$ as

$\begin{equation} k = f_\mathrm{sky} \sum_{\ell = \ell_0}^{\ell_1} (2 \ell + 1) . \end{equation}$

If we only have the BB error bars, $$\sigma \left( \mathcal{C}_{\ell,\mathrm{obs}}^{BB} \right)$$, then we can't solve for both $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$. However, if we assume that EE and BB have symmetric noise (but very different signal power spectra), then we can solve the following quadratic expression for $$\mathcal{N}_\ell$$.

$\begin{equation} 0 = \left( \sigma_{EE}^2 - \sigma_{BB}^2 \right) \mathcal{N}_\ell^2 + 2 \left( \mathcal{C}_{\ell,\mathrm{theory}}^{BB} \sigma_{EE}^2 - \mathcal{C}_{\ell,\mathrm{theory}}^{EE} \sigma_{BB}^2 \right) \mathcal{N}_\ell + \left( \mathcal{C}_{\ell,\mathrm{theory}}^{BB} \sigma_{EE} \right)^2 - \left( \mathcal{C}_{\ell,\mathrm{theory}}^{EE} \sigma_{BB} \right)^2 . \label{eq:Nl_quad} \end{equation}$

For the following experiments, I use the available data to solve for $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ (where possible), then fit the results to functional forms.

\begin{align} \mathcal{N}_\ell \mathcal{B}_\ell^2 &= \mathcal{N} \left[ 1 + \left( \frac{\ell}{\ell_\mathrm{knee}} \right)^\alpha \right] \\ f_\mathrm{sky} &= f \left[ 1 - \exp \left( - \ell / \ell_\mathrm{filt} \right) \right] \end{align}

The functional form for $$\mathcal{N}_\ell$$ is familiar and widely used. The functional form for $$f_\mathrm{sky}$$ is just something that I made up to account for filtering that decreases the number of effective modes at low $$\ell$$. I wanted a function that would make $$f_\mathrm{sky} \rightarrow 0$$ for $$\ell \rightarrow 0$$ while only adding one extra parameter and also ensuring that $$f_\mathrm{sky}$$ remains positive.

BICEP2 / Keck Array

The Figure 1 pager lets you view results for BICEP2 150 GHz (PRL 112, 24, 241101, 2014), BKP 150 GHz (PRL 114, 10, 101301, 2015), BK14 95 and 150 GHz (PRL 116, 3, 031302, 2016), and BK15 95, 150, and 220 GHz (submitted to PRL). For BICEP2 and Keck Array, I have access to our ensembles of noise-only simulations, so I can calculate $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ directly from the mean and variance of the bandpowers. However, this is also a good opportunity to check the reliability of the quadratic $$\mathcal{N}_\ell$$ solution given in equation $$\ref{eq:Nl_quad}$$ above. Clicking through the pager, you can see that the “internal noise sims” version yields higher $$\mathcal{N}_\ell$$ but also larger $$f_\mathrm{sky}$$. This is due to the fact that post-apodization BICEP/Keck maps have larger effective area for noise than for signal. The survey weight-like combination, $$f_\mathrm{sky} / \mathcal{N}_\ell$$, does come out ∼10–20% higher for most of the data release options when we use the internal noise sims.

One additional note for BICEP/Keck results is that the 95 GHz beam size listed in the BK14 and BK15 papers (43 arcmin FWHM) does not do a good job of flattening the $$\mathcal{N}_\ell$$ curve at high $$\ell$$. I used 47 arcmin for BK14 95 GHz and 49 arcmin for BK15 95 GHz, which work fairly well by eye. The published beam widths for 150 and 220 GHz (30 and 20 arcmin FWHM, respectively) work well without modification. Figure 1: $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ corresponding to BICEP/Keck publications. For the $$\mathcal{N}_\ell$$ figures, fit parameters $$\mathcal{N}$$, $$\ell_\mathrm{knee}$$, and $$\alpha$$ are listed in the figure title. For the $$f_\mathrm{sky}$$ figures, fit parameters $$f$$ and $$\ell_\mathrm{filt}$$ are listed in the figure title.

QUIET

The Figure 2 pager lets you view results for QUIET 43 GHz (ApJ 741, 2, 111, 2011) and 95 GHz (ApJ 760, 2, 145, 2012). Both of these results combine from four fields that are each ∼250 deg2. I used bandpower error bars from the Pseudo-$$\mathcal{C}_\ell$$ pipeline, which can be obtained from LAMBDA, for both frequencies. Signal bandpower expectation values were calculated from a lensed-ΛCDM model, ignoring the marginal synchrotron detection that was reported for QUIET patch 2a at 43 GHz.

The quadratic solution for $$\mathcal{N}_\ell$$ relies on there being a significant difference between $$\left( \mathcal{C}_{\ell,theory}^{EE} + \mathcal{N}_\ell \right)$$ and $$\left( \mathcal{C}_{\ell,theory}^{BB} + \mathcal{N}_\ell \right)$$. Since QUIET had relatively low sensitivity, this estimate is very noisy. What I did instead was to calculate the quadratic solution, use it to estimate $$f_\mathrm{sky}$$ by eye (assuming $$f_\mathrm{sky}$$ constant with $$\ell$$), then use this value to derive $$\mathcal{N}_\ell$$ from the bandpower error bars as in equation $$\ref{eq:sigma}$$. By this method, I obtained $$f_\mathrm{sky}$$ = 0.025 for 43 GHz and 0.015 for 95 GHz (1000 deg2 corresponds to 0.024). The $$f_\mathrm{sky}$$ plots in Figure 2 show the quadratic solution results while the $$\mathcal{N}_\ell$$ plots assume fixed $$f_\mathrm{sky}$$. Figure 2: $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ corresponding to QUIET publications. For the $$\mathcal{N}_\ell$$ figures, fit parameters $$\mathcal{N}$$, $$\ell_\mathrm{knee}$$, and $$\alpha$$ are listed in the figure title. For the $$f_\mathrm{sky}$$ figures, fit parameters $$f$$ and $$\ell_\mathrm{filt}$$ are listed in the figure title.

ABS

The Figure 3 pager lets you view results for ABS 150 GHz (submitted to JCAP). Error bars for EE and BB bandpowers were copied from Table 4 (I simply averaged the upper and lower error bars). There is some potential inaccuracy because my method assumes Monte Carlo error bars but the ABS EE and BB error bars are obtained by a fit to a likelihood function (see Section 6.1 of the ABS paper). Note that ABS results are reported for only one of the three CMB fields that were observed (Field A), but this field received the majority of the observing time.

I used the same trick for ABS as for QUIET—the quadratic solution for $$f_\mathrm{sky}$$ looks approximately consistent with 0.03 (but 2400 deg2 corresponds to $$f_\mathrm{sky}$$ = 0.058). Figure 3: $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ corresponding to the ABS 2018 publication. For the $$\mathcal{N}_\ell$$ figures, fit parameters $$\mathcal{N}$$, $$\ell_\mathrm{knee}$$, and $$\alpha$$ are listed in the figure title. For the $$f_\mathrm{sky}$$ figures, fit parameters $$f$$ and $$\ell_\mathrm{filt}$$ are listed in the figure title.

POLARBEAR

The Figure 4 pager lets you view results for POLARBEAR one-year (ApJ 794, 2, 171, 2014) or two-year (ApJ 848, 2, 121, 2017) BB publications. The one-year results are available from LAMBDA; I copied the two-year results from Table 4 of that paper (simply averaged upper and lower error bars).

POLARBEAR only published their BB bandpower and error bars, so I couldn't make a quadratic solution for $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$. Instead, I took the total sky area to be 25 deg2 as given (corresponds to $$f_\mathrm{sky}$$ = 0.00061) and solved for $$\mathcal{N}_\ell$$ using equation $$\ref{eq:sigma}$$.

The quoted beamsize for POLARBEAR at 150 GHz is 3.5 arcmin FWHM, but this did not do a good job of flattening $$\mathcal{N}_\ell$$ at high $$\ell$$. Instead, I used 4.3 and 4.7 arcmin FWHM for the one-year and two-year results, respectively.

Some notes from Yuji Chinone about POLARBEAR noise:

We have validated our noise is white in TOD for our science band ($$500 \lt \ell \lt 2100$$). But we saw a $$1/f$$ noise in $$\Delta \mathcal{C}_\elll^{BB}$$ because of the size of our patch (very small) and mode loss due to the filterings. We think we could achieve an ell knee down to 170, but our null test might suggest a knee around 300–400 if we extend the science band smaller. This number is a little bit smaller than yours, but it seems roughly consistent (I did a fit w/ $$\Delta \mathcal{C}_\ell$$ w/ smaller bin size internally.) Note that this is from a result without HWP modulation (as you know.)

My takeaway is that a significant part of what I am attributing to excess low-$$\ell$$ noise is really mode loss from the very small POLARBEAR scan regions, but without the $$EE$$ error bars I have no ability to disentangle the two effects. Figure 4: $$\mathcal{N}_\ell$$ corresponding to POLARBEAR one-year and two-year publications. Fit parameters $$\mathcal{N}$$, $$\ell_\mathrm{knee}$$, and $$\alpha$$ are listed in the figure title.

ACTpol

The Figure 5 pager lets you view results for ACTpol one-year (JCAP 10, 007, 2014) and two-year (JCAP 06, 031, 2017) publications. For both seasons, the bandpower error bars and associated data can be extracted from the CMB likelihood modules available on LAMBDA.

The ACT $$EE$$ and $$BB$$ spectra are calculated with a cross-spectrum analysis from four maps, constructed using alternating observing days. This results in slightly larger error bars than would be obtained from an auto-spectrum analysis. The results here show the expected $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ that would be obtained for an auto-spectrum (under the assumption that the four maps have equal weight). Many of the other experiments shown in this posting also use cross-spectrum analyses, but with a much larger set of maps such that the statistical difference between the auto-spectrum and cross-spectrum is negligible.

The quadratic method produces nice results for ACTpol $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$, but I limited the analysis to bins with $$\ell \le 2000$$. For the one-year result, $$f_\mathrm{sky}$$ has a value of 0.0028 and is flat with $$\ell$$. For the two-year result, $$f_\mathrm{sky}$$ shows a decreasing linear trend with $$\ell$$ (which doesn't match my functional form) and $$\mathcal{N}_\ell$$ comes out with significantly higher $$\ell_\mathrm{knee}$$ but shallower $$\alpha$$ slope. I don't have any good explanation for these differences. The ACTpol 150 GHz beamsize of 1.3 arcmin FWHM is small enough that minor changes don't have any significant effect on these results. Figure 5: $$\mathcal{N}_\ell$$ and $$f_\mathrm{sky}$$ corresponding to the ACTpol one-year and two-year publications. For the $$\mathcal{N}_\ell$$ figures, fit parameters $$\mathcal{N}$$, $$\ell_\mathrm{knee}$$, and $$\alpha$$ are listed in the figure title. For the $$f_\mathrm{sky}$$ figures, fit parameters $$f$$ and $$\ell_\mathrm{filt}$$ are listed in the figure title.

SPTpol

The Figure 5 pager lets you view results for the SPTpol 100 deg2 BB publication (ApJ 807, 2, 151, 2015) at 95 and 150 GHz. I used bandpower error bars and associated data extracted from the CosmoMC likelihood module that is available on LAMBDA.

Since only BB bandpowers are available for this publication, I couldn't use the quadratic method. Instead I assumed sky area of 100 deg2 ($$f_\mathrm{sky}$$ = 0.0024) and used equation $$\ref{eq:sigma}$$. It might be possible to include the EE results from Crites et al, 2014, to solve for $$f_\mathrm{sky}$$, but that paper uses different $$\ell$$ bins and could include other important differences in the data analysis. Figure 6: $$\mathcal{N}_\ell$$ corresponding to the SPTpol 100 deg2 publication. Fit parameters $$\mathcal{N}$$, $$\ell_\mathrm{knee}$$, and $$\alpha$$ are listed in the figure title.

Not included here

I haven't tried to analyze BICEP1 or QUaD. BICEP1 probably isn't too interesting, since we already have an abundance of BICEP/Keck results. QUaD might be useful as an additional mid-sized aperture, to compare with POLARBEAR.