In the past years, the LHC has collected an impressive amount of data and placed constraints on a multitude of models for new physics. Most of these searches are targeting high-pT signatures corresponding to new strongly interacting heavy particles. However, up to now, there has been no discovery of any elementary beyond the standard model (BSM) particle, which has motivated the community to consider a broader range of different physics signatures. In particular, if new particles are light and weakly coupled, they could travel a macroscopic distance before decaying. Searches for such long-lived particles (LLPs) are experimentally clean and theoretically well motivated (see, e.g., Ref. [1]). While current and planned searches for MeV to GeV range LLPs typically employ beam-dump experiments or meson factories, LLPs would also be produced abundantly in high-energy pp collisions at the LHC. However, they are typically produced with low pT and therefore will move along the beam pipe and escape detection in the ATLAS [2] and CMS [3] experiments.

In recent papers [4,5], we proposed a new experiment searching for LLPs, the Forward Search Experiment (FASER) at the LHC. FASER would be placed in the forward direction from the ATLAS or CMS interaction point (IP) and operate concurrently with the ongoing high-pT searches. In particular, we considered a small size detector (20 cm to 1 m radius and a 10 m long cylinder) placed in a representative location 400 m away from the IP along the beam axis, possibly in a side tunnel, after the main LHC tunnel enters the arc section and starts to curve. Importantly, the existing LHC infrastructure, including neutral absorbers and magnets deflecting charged particles, would provide natural shielding from various standard model (SM) backgrounds. An additional layer of shielding is provided by the rock and concrete that separate the main tunnel from the side tunnel in which FASER would be placed.

A more detailed study of the LHC infrastructure around the ATLAS IP revealed an attractive location for FASER in the currently unused side tunnel TI18 in the distance about 480 m away from the IP. It is a former service tunnel used by large electron-positron (LEP) collider as a connection between the super proton synchrotron and the main tunnel. In the following, we will perform sensitivity studies for this location.

In this study, we analyze FASER’s prospects for discovering heavy neutral leptons (HNLs) as a well-known example of new fermionic particles that can be found in forward searches [6–10]. This analysis is complementary to our previous studies. In particular, unlike dark photons, HNLs with mass above ∼1  GeV can be abundantly produced in heavy meson decays thanks to their mixing with the SM (active) neutrinos. However, in contrast to dark Higgs bosons, HNLs couplings to mesons are not dictated by the quark Yukawa couplings but rather by the set of unknown Yukawa couplings between HNLs and the active neutrinos, as well as by the Cabibbo-Kobayashi-Maskawa (CKM) mixing parameters. As a result, HNLs can also be efficiently produced in D-meson decays, while less abundantly produced B mesons play a dominant role only when other channels are kinematically forbidden.

Alternatively, one can also consider a detector in some near location inside the straight intersection of the main LHC tunnel between the two beam pipes after they split. The location that is as close as possible to the IP is right behind the neutral particle absorber Target Absorber Neutral (TAN), which we chose as a representative case in Refs. [4,5]. The expected signal of new physics in this case can be enhanced with respect to the far location in models in which the lifetime of the LLPs is too small to reach the far location. This is, e.g., true for a dark photon search if its kinetic mixing with the SM photon is not suppressed too much as discussed in Ref. [4]. On the other hand, for a sufficiently large lifetime, this advantage is missing, while the far location suffers much less from the expected background and can additionally benefit from allowing more space for a larger detector as we illustrated in the case of dark Higgs boson [5].

As we will see below, the lifetime of HNLs is typically large, and they can easily overshoot the detector at the near location once additionally boosted. In the following, we will therefore focus exclusively on the detector placed at far location in the TI18 tunnel.

The paper is organized as follows. In Sec. II, we review the basic properties of HNLs. We discuss possible production channels of sterile neutrinos at the LHC as well as their decays that give rise to a signal in FASER in Sec. III. The main results of this study with the sensitivity reach of FASER in the searches for HNLs are presented in Sec. IV. Section V is devoted to discussion of selected scenarios going beyond the minimal seesaw mechanism that can be probed by FASER. We conclude in Sec. VI. A more detailed discussion of fragmentation functions for D and B mesons can be found in Appendix.

Undoubtedly, the most important reason to add new heavy neutral leptons to the SM is to explain the neutrino oscillations [11,12] since they provide an elegant way to generate nonzero neutrino masses via the seesaw mechanism [13–17]. In type-I seesaw models, one extends the SM by adding N neutral right-handed fermions (identified with HNLs) that couple to the SM neutrinos similarly to the coupling between left- and right-handed components of the charged leptons. Since additional right-handed fermions are SM singlets, also Majorana mass terms are allowed, leading to the following Lagrangian, L=LSM+iN˜¯I∂N˜I-FαIL¯αN˜IΦ˜-12N˜¯IcMIN˜I+H.c.,(1)where N˜I=1,…,N are the right-handed HNLs and we will assume N=3, Φ˜i=εijΦj*, Φ, and Lα=e,μ,τ are Higgs and lepton doublets; FαI are the Yukawa couplings between HNLs and the SM leptons; and M=diag(M1,M2,M3) is the Majorana mass matrix for HNLs.

After electroweak symmetry breaking (EWSB), the Higgs field gets a nonzero vacuum expectation value, v=174  GeV, and terms in Eq. (1) proportional to Yukawa couplings generate mixing between HNLs and active neutrinos. The full 6×6 neutrino mass matrix reads Mν=[0MDMDTM],(2)where (MD)αI=vFαI. One then obtains nonzero masses for the SM neutrinos after diagonalization.

Similarly, the HNL mass eigenstates, NI (denoted without a tilde), get small contributions from active states with mixing angles given by UαI2≃|FαI|2v2MI2,(3)where typically UαI≪1. HNLs inherit their interactions with the SM particles from the active neutrinos with couplings suppressed by the mixing angles. In particular, the charged current interaction for the HNL is given by -(g/2)Wμℓ¯L,αγμUαINI+H.c.. On the other hand, their masses, mNI, are mostly governed by Majorana masses MI with only small corrections from mixing. Hence, they are often referred to as sterile neutrinos or simply heavy neutrinos.

If sterile neutrinos are the only particles added to the SM to explain the active neutrino masses, then the presence of at least two heavy neutrinos is required to generate two measured mass differences between the active neutrinos. At least three heavy neutrinos are required if also the lightest active neutrino winds up being massive. In this case the HNL sector has a three-generation structure similarly to the SM.

However, not all these sterile neutrinos are necessarily within the reach of FASER. Hence, when discussing the sensitivity reach, we will focus for simplicity on a single sterile neutrino, N, with mass mN and mixing angles with the active neutrinos denoted by UeN, UμN, and UτN for the electron, muon, and tau neutrino, respectively. In addition, for the purpose of presenting results, we will assume that N mixes exclusively with only one of the active neutrinos. We will therefore study the reach of FASER for three scenarios in which the mixing is dominated by only one nonzero value of UℓN in each case, where ℓ=e, μ, τ. On the other hand, the reach for scenarios in which more than one HNL has mass and couplings in the region of the parameter space covered by FASER, or when the mixing between N and active neutrinos has a more complicated pattern, can be deduced from our results based on the plots presenting the number of events in each of the three cases considered by us. The effective parameters that we vary are then the following: mN,UeN,UμN,UτN,where only one UℓN≠0  at  a  time.(4)

The allowed region for the mass of sterile neutrinos spans over more than 15 orders of magnitude (see, e.g., Ref. [9] for a recent review). Theoretically, the most appealing, but experimentally extremely challenging, are seesaw models with MI close to the scale of Grand Unified Theories (GUTs) (see, e.g., Ref. [18]) that allow one to naturally fit the experimental neutrino data with Yukawa couplings FαI∼1. For much lower MI, one naively expects FαI≪1, which would keep sterile neutrinos beyond the reach of low-energy experiments. However, for more than one generation of HNLs, accidental or symmetry driven cancellations between contributions to the active neutrino mass matrix coming from different NIs allow larger values of Yukawa couplings to be considered in Eq. (3) [19–24]. As a result, one can effectively study HNLs with the NI masses and mixing angles as independent parameters, while still preserving the primary motivation for such models that come from explaining the active neutrino data.

In order to explain the masses of active neutrinos via the seesaw mechanism, the mixing parameters UℓN need to be sufficiently large. If light enough, sterile neutrinos could then thermalize in the early Universe and distort successful predictions of the big bang nucleosynthesis (BBN) by contributing to the number of relativistic degrees of freedom [25]. They could also cause an additional entropy production from their late-time decays. This can be circumvented, if HNLs decay before BBN. Combining this and current experimental bounds, one obtains an effective lower limit on the sterile neutrino mass mN≳140  MeV in the range of the mixing angles that can be covered by searches for LLPs. More detailed studies [26–28] show that in some cases mN can be lowered down to O(10  MeV).

On the other hand, BBN bounds can be evaded for an even much lighter sterile neutrino, provided that its mixing angles with the active neutrinos are suppressed. This gives rise to a sterile neutrino dark matter (DM) candidate with mass at the ∼keV scale that was, e.g., proposed as a possible explanation to a 3.5 keV excess [29,30] in the XMM-Newton data. One can both accommodate the observed active neutrino data, as well as keep N1 as a DM candidate, if heavier HNLs, N2,3, have larger mixing angles and satisfy the aforementioned lower mass limit from BBN. In particular, this is true for the renowned neutrino minimal standard model (νMSM model) [24]. In such a case, while an effectively stable N1 would remain outside the reach of FASER, the displaced decays of heavier sterile neutrinos could give rise to an observable signal. For other models motivating the search for GeV-scale sterile neutrinos see, e.g., Refs. [31,32].

One additional motivation that lies behind low-scale seesaw models is that, similarly to high-scale models, they can also explain the baryon asymmetry of the Universe via thermal leptogenesis [33]. The lepton asymmetry in this case is generated from CP violation in out-of-equilibrium production and subsequent evolution of sterile neutrinos with oscillation effects taken into account. This requires low values of Yukawa couplings FαIs. As a result, the mixing angles are typically of order UℓN∼10-5-10-3 (see, e.g., Ref. [34] for an extensive discussion) and lie beyond the reach of current experiments. A new generation of dedicated LLP searches, including FASER, is therefore needed to study this scenario.

Sterile neutrinos can be produced at the LHC in decays of mesons and tau leptons [6], heavy baryon decays [35], as well as in production from on- or off-shell gauge or Higgs bosons (see, e.g., Refs. [36,37] for a recent discussion). Given our focus on very forward-going light HNLs, the main production mechanism is via meson decays. The dominant contribution comes from leptonic and semileptonic decays of pseudoscalar D and B mesons, while decays of vector mesons are subdominant [6]. In addition, tau decays to HNLs are taken into account for a scenario in which a sterile neutrino mixes exclusively with the tau neutrino. Kaon decays into sterile neutrinos are kinematically allowed only in the mass range at which strong constraints exist from previous beam-dump experiments. Hence, they will not play important role in our analysis. We do not consider charged pion decays into HNLs since they only become possible for mN≲mπ, which is disfavored by strong cosmological bounds, as discussed above. Importantly, although two-body leptonic decays of mesons are chiral suppressed in the SM, this suppression is partially overcome if the sterile neutrino mass is heavy enough since its mass, mN, replaces the mass of a charged lepton in the chiral suppression factor [38].

Contributions from baryon decays are subdominant in the case of charmed baryons and typically also small for bottom baryons, although in some regions of parameter space, they can be as large as 15% of the total production rate [35]. We will neglect them hereafter given larger uncertainties that are expected in our signal rates.

The expected signal from sterile neutrinos decaying in FASER consists of two simultaneous high-energy charged tracks, which originate from a vertex inside the detector and the combined momentum of which points into the direction of the IP. While detailed background analysis goes beyond the scope of this paper, one can argue that natural and infrastructure-based shielding, as well as specific properties of the signal, should allow one to disentangle it from background [4]. In particular, SM particles produced abundantly at the IP would be either deflected by the LHC magnets or stopped by neutral absorbers before they reach FASER. On the other hand, energetic particles produced in beam-gas collisions in the beam pipe close to the detector would effectively lose their energy in the rock and concrete separating the side tunnel from the main tunnel. The kinematic features of the signal, such as the directionality of the charged particles observed in FASER, would provide an additional handle that could help to discriminate between these backgrounds and the signal events, as well as to reject events with cosmic-ray origin. In the following, we will for simplicity assume zero background when discussing FASER’s sensitivity.

FASER’s sensitivity reach can also be affected by the finite detection efficiency. In particular, we correct the expected number of events to take into account sterile neutrino decays into three SM neutrinos. In addition, the reconstruction of the direction of visible tracks in the detector might be affected by possible multibody decays of the HNLs and the presence of invisible active neutrinos in the final state. However, given a large boost of the HNL and its decay products, we expect that such reconstruction should be sufficiently good to help disentangle between background and signal events. In the following, we will assume 100% efficiency for other decay channels, while a more detailed analysis of this effect is postponed to a later dedicated study.

In Figure 5, we show the sensitivity reach of FASER to search for sterile neutrinos that mix only with the electron neutrino, where the gray bands correspond to the current exclusion limits. In the left panel, we show the sensitivity of the R=1  m detector for the scenario with UeN≠0 decomposed into contributions from kaon (blue), D-meson (green), and B-meson (red) decays. Typically lighter-meson decays dominate when kinematically allowed since they are produced more abundantly at the LHC. The solid lines correspond to a fixed number of signal events, n=3,10,102,103,104. As can be seen, one can expect up to ∼1000 events with HNL origin in FASER for mN≳mD and ∼10 signal events for sterile neutrinos produced in D-meson decays. Clearly, possible changes in the number of signal events by a factor O(1) from a finite efficiency of the detector or non-negligible background would have a mild impact on the reach especially in the case of B-meson decays. The region most sensitive to such changes corresponds to mN>mB-mD where dominant semileptonic decays of B mesons into HNL and D mesons are kinematically forbidden. Instead, a few events in this mass regime can be expected from leptonic B± and Bc± decays that are otherwise subdominant. This results in a characteristic bumplike shape of the sensitivity line.

In the right panel of Figure 5, we compare the sensitivity of the R=20  cm and R=1  m detector for 3  ab-1 integrated luminosity. As can be seen, the reach for R=1  m is significantly improved with respect to the one obtained for a smaller detector. This is not a surprise since HNLs, similarly to dark Higgs bosons [5], are typically produced in heavy-meson decays and, therefore, they are less collimated around the beam axis than, e.g., dark photons produced in pion decays [4]. The reach could be further improved, especially for mN≲mD, for an even larger radius, as can be deduced from the bottom right panel of Figure 1.

Similarly, the cases of mixing with the muon neutrino and tau neutrino are shown in Figs 6 and 7, respectively. As can be seen, FASER will probe parts of the yet unconstrained region of the parameter space in each of these scenarios. In the electron and muon case, current bounds below the kaon threshold are stronger than the reach of FASER. However, FASER’s capability is much improved once mN grows and the typical decay length of boosted sterile neutrinos becomes comparable to the distance between the IP and the detector (compare Figure 3). On the other hand, the scenario with nonzero mixing with the tau neutrino is much less constrained. This leads to an even larger region of the parameter space for HNLs that has not been probed yet and is characterized by an excellent discovery potential in FASER with possibly even tens of thousands of expected signal events for 3  ab-1 integrated luminosity. Importantly, the reach for mN>mB-mD-mτ corresponds to the leptonic decays of B± and Bc± and leads to a bumplike shape of the sensitivity line which is much more pronounced than in the case of nonzero mixing with the electron and muon neutrinos.

In the range of mN relevant for FASER, the most stringent bounds on sterile neutrino mixing with the electron neutrino or muon neutrino come from past and present beam-dump experiments at CERN (PS191 [62], CHARM [63], and NA62 [64]) and IHEP-JINR [65]. For mN≳2  GeV, the strongest limits come from the search for B decays into HNLs at Belle II [66] and from limits on the Z boson decays into HNLs from the LEP data collected by the DELPHI Collaboration [67]. For mixing with the muon neutrino, other important bounds come from search for a double-peak structure in K→μν decays [68] and the NuTeV beam-dump experiment [69]. In the case of mixing with the tau neutrino, current limits are much weaker with the leading bounds coming from the CHARM [70] and DELPHI [67] collaborations. If the mixing angles become low enough, strong bounds from BBN [26–28] constrain the parameter space of HNLs from below.

In the scenario with nonzero mixing with the electron neutrino, UeN≠0, other important bounds come from null searches of the neutrinoless double-beta decay, denoted as 0νββ. The most stringent current limit on the 0νββ decay half-life T1/20ν≳1.07×1026  yr comes from a combined analysis of the phase-I and phase-II data acquired by the KamLAND-Zen experiment [71]. This can be translated (see, e.g., Refs. [50,72]) into an approximate limit on the mixing angle |UeN|2/mN≲2.1×10-8  GeV-1. However, as discussed above, more than one sterile neutrino is required for the seesaw mechanism to generate correct active neutrino masses. In addition, some of the sterile neutrinos can be lighter than the typical momentum transfer q∼100  MeV in the 0νββ process provided they interact weakly enough so as to not alter BBN. In this case, additional cancellations between various contributions to the 0νββ rate may occur and effectively weaken the corresponding bound [73]. It is therefore not shown in Figure 5.

For comparison, in Figure 5, we also show the expected sensitivities for sterile neutrino searches in the proposed SHiP detector [56], the DUNE [57] and NA62 [58] experiments, and the LHC searches for a prompt lepton plus a single displaced lepton jet [59] (see also Ref. [74] for a relevant study for the LHCb). As can be seen, FASER has comparable sensitivity to NA62 for sterile neutrinos produced in D-meson decays. This region in the parameter space for both electron and muon mixing will also be probed by the future DUNE facility for which we show results following Ref. [56]. They correspond to the analysis performed for the five years of data taking by the 30 m long-baseline neutrino experiment (LBNE) near detector with 5×1021 protons on target and assuming a normal hierarchy of neutrinos.

Above the D-meson threshold, the abundant production of forward B mesons at high-energy pp collisions at the LHC works in favor of FASER. This allows FASER to probe the region between the aforementioned planned searches and the expected reach of the LHC searches [59] as shown for the scenario with UμN≠0 in the left panel of Figure 6. The LHC searches will also be sensitive to other scenarios with nonzero UeN or UτN. However, as discussed in Ref. [59], dedicated analysis of relevant backgrounds and tagging efficiency would have to be performed by experimental collaborations to obtain solid results. Therefore, they are not shown in Figs. 5 and 7, where we present the reach for the UeN≠0 and UτN≠0 cases, respectively.

In case of nonzero mixing with the tau neutrino, UτN≠0, HNL production in D-meson decays is restricted to mN<mD-mτ. The main contribution comes from the decay Ds+→τN. Larger masses mD-mτ<mN≲mτ can be probed by hadronic and leptonic tau decays, τ→Nπ, NK+, Nρ+ and τ→Nℓνℓ. Here, the τs are mainly produced via the decay Ds+→τν. This region of the parameter space can also be partially probed by searches based on τ production in B factories [60] and their subsequent decays into sterile neutrinos, as well as by search for double-cascade events initiated by high-energy neutrinos up-scattering into sterile neutrinos in the IceCube detector [61].

The strongest expected reach in the range of the HNL masses probed by FASER could be obtained by the proposed SHiP detector. In order to compare the reach of FASER to the one of SHiP, it is useful to note that to a good approximation the number of expected events scales like Nev∼NHNL×PNdet∼|UℓN|4×Nmes×ΔEmes,(7)where in the second step we used Eq. (5) and for simplicity we neglected the geometrical acceptance of the detector. The expected number of mesons produced for a given experiment is denoted by Nmes, while Emes is the typical energy of mesons that give rise to sterile neutrinos decaying within the volume of each detector. For both considered experiments, the relevant numbers read SHiP:  ND∼3.4×1017,NB∼3.2×1013,Emes∼25  GeV,Δ∼50  m,FASER:  ND∼1.9×1016,NB∼1.4×1015,Emes∼1  TeV,Δ∼10  m.(8)Combining Eqs. (7) and (8), one obtains for SHiP a number of events with B-meson origin only about five times larger. Given that in the regime of large lifetime the number of expected events scales like Nev∼|UℓN|4, one obtains only slightly better sensitivity (by a factor of ≲2) of SHiP to sterile neutrinos produced in B-meson decays in the (mN,UℓN) plane. In addition, for somewhat heavier Bc± mesons, FASER can gain even more in meson production from much larger collision energy than SHiP. As a result, the sensitivity of both detectors is comparable at the largest accessible values of mN∼4  GeV. The loss of sensitivity for these large masses and increasing mixing angles to values UℓN≳10-3 can be understood since then the sterile neutrino lifetime becomes too low for HNLs to reach FASER.

On the other hand, production of D mesons is less suppressed for the SHiP center-of-mass energy 27 GeV, and therefore larger luminosity allows one to obtain ∼103–104 more HNL events with D-meson origin in SHiP than in FASER. Hence, the reach in the mixing angle is worse for FASER by about an order magnitude, as can be seen in Figure 5. This is also true for sterile neutrinos produced in τ decays, as seen in Figure 7, since they typically come from D-meson decays.

In previous sections, we have analyzed in details FASER’s sensitivity reach to HNLs described effectively by only their mass and the mixing angles with the active neutrinos. We will now discuss how this search can be relevant to selected more complex scenarios going beyond the minimal seesaw mechanism with only right-handed neutrinos added to the SM. In particular, the reach for historically very important left-right symmetric models is discussed in Sec. V A. In Sec. V B, we illustrate the possible connection of the sterile neutrino search in FASER to DM and the baryon asymmetry of the Universe.

Searches for light long-lived particles have recently become widely accepted as one of the most important aims to achieve in the next generation of dedicated experiments looking for BSM physics. Among many models of new physics that can be probed in such searches, one of the most extensively studied scenarios predicts heavy neutral leptons with masses of order GeV and small mixing angles with the active neutrinos. In particular, HNLs appear naturally in the context of the seesaw mechanism that generates nonzero neutrino masses in the SM. Although typically the seesaw scenario predicts sterile neutrinos with large masses of order the GUT scale, it is also possible to obtain light HNLs in theoretically appealing models.

In this study, we have analyzed the expected sensitivity reach to HNLs of the newly proposed detector to be placed along the LHC beam axis, named Forward Search Experiment, or FASER [4,5]. This analysis is complementary to our previous studies for dark photons that are mainly produced in pion decays and for dark Higgs bosons that typically come from B-meson decays. In the case of HNLs, also decays of D mesons play an important role, while B mesons start to dominate only when other channels are kinematically forbidden.

We compare the sensitivity reach of FASER in the HNL parameter space to other proposed detectors. In particular, we show that for sterile neutrinos produced mainly in B-meson decays even a relatively small cylindrical detector with ∼10  m length and R≲1  m radius can have reach comparable to the proposed SHiP detector, while other experiments typically lose their sensitivity in this range of the HNL mass. In this mass regime, the current bounds in the mixing angles could be improved by about an order of magnitude. On the other hand, the reach in the mixing angle for HNLs produced in D-meson decays is comparable to the one expected from the NA62 experiment and can be surpassed by SHiP and DUNE due to their much larger luminosities.

However, even in the case of D-meson decays, FASER can probe an interesting region in the HNL parameter space. In particular, as discussed in Sec. V B, it corresponds to the attractive cosmological scenario in which sterile neutrinos emerge from the mirror sector of the SM with heavy right-handed neutrinos as mediators. In this scenario, the search for sterile neutrinos in FASER could shed light on other important questions in contemporary physics including the nature of DM and the origin of baryon asymmetry of the Universe. It could also be related to the increasingly popular solution to the hierarchy problem via neutral naturalness.

Importantly, looking for a signal from light sterile neutrino decays in FASER will play a role complementary to searches for heavier HNLs in high-pT oriented experiments at the LHC. We illustrate this in Sec. V A for the case of the left-right symmetric models that provide a natural extension of the SM gauge group to explain the asymmetry between the left and right fermions in the SM.

Last, but not least, in scenarios with sterile neutrinos emerging from the mirror sector of the SM, the photon of the mirror sector mixes with the SM photon [91,94]. It can then effectively play the role of the dark photon. Its mass and mixing parameter would then typically be within the reach of FASER discussed in our previous study [4]. In this case, one expects to simultaneously see events with the HNL and dark photon origin with possibly distinct final states of their decays.