<?xml version="1.0" encoding="UTF-8"?>
<article xmlns="http://specifications.silverchair.com/xsd/article/1/0/SCJATS-journalpublishing1-0.xsd" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">ptep</journal-id>
<journal-title-group>
<journal-title>Progress of Theoretical and Experimental Physics</journal-title>
</journal-title-group>
<issn pub-type="epub">2050-3911</issn>
<publisher>
<publisher-name>Oxford University Press</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.1093/ptep/ptx175</article-id>
<article-id pub-id-type="publisher-id">ptx175</article-id>
<article-id pub-id-type="arxiv">arXiv:1710.04811</article-id>
<article-categories>
<subj-group subj-group-type="category-toc-heading">
<subject>Papers</subject>
<subj-group subj-group-type="category-toc-heading">
<subject>Theoretical Particle Physics</subject>
</subj-group>
</subj-group>
<subj-group subj-group-type="category-journal-collection">
<subject>PTEP/B40</subject>
<subject>PTEP/B41</subject>
<subject>PTEP/B42</subject>
<subject>PTEP/B43</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Electroweak symmetry breaking and mass spectra in six-dimensional gauge&#x2013;Higgs grand unification</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Hosotani</surname><given-names>Yutaka</given-names></name>
<xref ref-type="aff" rid="AFF1"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Yamatsu</surname><given-names>Naoki</given-names></name>
<xref ref-type="aff" rid="AFF2"/>
<xref ref-type="corresp" rid="COR1"/>
<email xlink:type="simple">yamatsu@cc.kyoto-su.ac.jp</email>
</contrib>
</contrib-group>
<aff id="AFF1"><italic>Department of Physics, Osaka University, Toyonaka, Osaka 560-0043, Japan</italic></aff>
<aff id="AFF2"><italic>Maskawa Institute for Science and Culture, Kyoto Sangyo University, Kyoto 603-8555, Japan</italic></aff>
<author-notes>
<corresp id="COR1">E-mail: <email>yamatsu@cc.kyoto-su.ac.jp</email></corresp>
</author-notes>
<pub-date pub-type="cover">
<month>02</month>
<year>2018</year>
</pub-date>
<pub-date pub-type="collection">
<day>01</day>
<month>02</month>
<year>2018</year>
</pub-date>
<pub-date pub-type="epub" iso-8601-date="2018-02-24">
<day>24</day>
<month>02</month>
<year>2018</year>
</pub-date>
<volume>2018</volume>
<issue>2</issue>
<elocation-id>023B05</elocation-id>
<history>
<date date-type="received">
<day>16</day>
<month>10</month>
<year>2017</year>
</date>
<date date-type="rev-recd">
<day>21</day>
<month>11</month>
<year>2017</year>
</date>
<date date-type="accepted">
<day>21</day>
<month>11</month>
<year>2017</year>
</date>
</history>
<permissions>
<copyright-statement>&#x000A9; The Author(s) 2017. Published by Oxford University Press on behalf of the Physical Society of Japan.</copyright-statement>
<copyright-year>2017</copyright-year>
<license license-type="cc-by" xlink:href="http://creativecommons.org/licenses/by/4.0/"><license-p>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<ext-link xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://creativecommons.org/licenses/by/4.0/" ext-link-type="uri">http://creativecommons.org/licenses/by/4.0/</ext-link>), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
<license-p>Funded by SCOAP<sup>3</sup></license-p>
</license>
</permissions>
<self-uri xlink:href="ptx175.pdf"/>
<abstract abstract-type="abstract"><title>Abstract</title>
<p>The mass spectra of the standard model particles are reproduced in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge&#x2013;Higgs grand unification in six-dimensional warped space without introducing exotic light fermions. Light neutrino masses are explained by the gauge&#x2013;Higgs seesaw mechanism. We evaluate the effective potential of the four-dimensional Higgs boson appearing as a fluctuation mode of the Aharonov&#x2013;Bohm phase <inline-formula><tex-math notation="LaTeX" id="ImEquation2"><![CDATA[$\theta_H$]]></tex-math></inline-formula> in the extra-dimensional space, and show that the dynamical electroweak symmetry breaking takes place with the Higgs boson mass <inline-formula><tex-math notation="LaTeX" id="ImEquation3"><![CDATA[$m_H \sim 125\,$]]></tex-math></inline-formula>GeV and <inline-formula><tex-math notation="LaTeX" id="ImEquation4"><![CDATA[$\theta_H \sim 0.1$]]></tex-math></inline-formula>. The Kaluza&#x2013;Klein mass scale in the fifth dimension is approximately given by <inline-formula><tex-math notation="LaTeX" id="ImEquation5"><![CDATA[$m_{\rm KK} \sim 1.230\,{\rm TeV}/\sin \theta_H$]]></tex-math></inline-formula>.</p>
</abstract>
<kwd-group kwd-group-type="jel">
<kwd>B40</kwd>
<kwd>B41</kwd>
<kwd>B42</kwd>
<kwd>B43</kwd>
</kwd-group>
<counts>
<page-count count="48"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="SEC1"><title>1. Introduction</title>
<p>The discovery of the last of the standard model (SM) particles, the Higgs boson, seems to imply that the non-vanishing vacuum expectation value (VEV) of the Higgs field spontaneously breaks the electroweak (EW) gauge symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation6"><![CDATA[$SU(2)_L\times U(1)_Y$]]></tex-math></inline-formula> to the electromagnetic gauge symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation7"><![CDATA[$U(1)_{\rm EM}$]]></tex-math></inline-formula>. Almost all observational data from low-energy experiments are consistent with the SM. The SM is a good low-energy effective theory. However, it is not clear whether or not the Higgs boson is a genuine fundamental scalar field, which usually suffer from the so-called gauge hierarchy problem.</p>
<p>There are several proposals to overcome the problem by making use of symmetries. One of them is the gauge&#x2013;Higgs unification. In this theory, the Higgs boson is identified with part of the extra-dimensional component of gauge fields in higher-dimensional spacetime [<xref ref-type="bibr" rid="B1">1</xref>&#x2013;<xref ref-type="bibr" rid="B15">5</xref>]. It is described as a four-dimensional (4D) fluctuation mode of the Aharonov&#x2013;Bohm (AB) phase <inline-formula><tex-math notation="LaTeX" id="ImEquation8"><![CDATA[$\theta_H$]]></tex-math></inline-formula> along the extra-dimensional space.</p>
<p>The <inline-formula><tex-math notation="LaTeX" id="ImEquation9"><![CDATA[$SU(2)_L\times U(1)$]]></tex-math></inline-formula> EW unification of the SM is formulated as the <inline-formula><tex-math notation="LaTeX" id="ImEquation10"><![CDATA[$SO(5)\times U(1)$]]></tex-math></inline-formula> gauge&#x2013;Higgs unification in the five-dimensional (5D) Randall&#x2013;Sundrum (RS) warped space [<xref ref-type="bibr" rid="B6">6</xref>&#x2013;<xref ref-type="bibr" rid="B16">16</xref>]. According to Refs. [<xref ref-type="bibr" rid="B11">11</xref>&#x2013;<xref ref-type="bibr" rid="B14">14</xref>], its phenomenology at low energies under the mass scale of the first Kaluza&#x2013;Klein (KK) modes is almost the same as in the SM for the AB phase <inline-formula><tex-math notation="LaTeX" id="ImEquation11"><![CDATA[$\theta_H \lesssim 0.1$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation12"><![CDATA[$Z'$]]></tex-math></inline-formula> bosons, which are the first KK modes of <inline-formula><tex-math notation="LaTeX" id="ImEquation13"><![CDATA[$\gamma$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation14"><![CDATA[$Z$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation15"><![CDATA[$Z_R$]]></tex-math></inline-formula>, are predicted around the <inline-formula><tex-math notation="LaTeX" id="ImEquation16"><![CDATA[$7 \sim 9\,$]]></tex-math></inline-formula>TeV range for <inline-formula><tex-math notation="LaTeX" id="ImEquation17"><![CDATA[$\theta_H=0.1 \sim 0.07$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation18"><![CDATA[$Z'$]]></tex-math></inline-formula> bosons can be produced at the 14 TeV LHC [<xref ref-type="bibr" rid="B15">15</xref>]. At electron&#x2013;positron linear colliders at energies of <inline-formula><tex-math notation="LaTeX" id="ImEquation19"><![CDATA[$250\,$]]></tex-math></inline-formula>GeV <inline-formula><tex-math notation="LaTeX" id="ImEquation20"><![CDATA[$\sim 1\,$]]></tex-math></inline-formula>TeV, the interference effects among <inline-formula><tex-math notation="LaTeX" id="ImEquation21"><![CDATA[$\gamma$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation22"><![CDATA[$Z$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation23"><![CDATA[$Z'$]]></tex-math></inline-formula> bosons give distinct signals of the gauge&#x2013;Higgs unification [<xref ref-type="bibr" rid="B16">16</xref>].</p>
<p>To incorporate the <inline-formula><tex-math notation="LaTeX" id="ImEquation24"><![CDATA[$SU(3)_C$]]></tex-math></inline-formula> gauge symmetry in higher-dimensional gauge theories and gauge&#x2013;Higgs unification scenario, grand unified theories (GUTs) based on a GUT gauge group <inline-formula><tex-math notation="LaTeX" id="ImEquation25"><![CDATA[$G_{\rm GUT}\ (\supset G_{\rm SM}:=SU(3)_C\times SU(2)_L\times U(1))$]]></tex-math></inline-formula> have been discussed in Refs. [<xref ref-type="bibr" rid="B17">17</xref>&#x2013;<xref ref-type="bibr" rid="B37">37</xref>]. The <inline-formula><tex-math notation="LaTeX" id="ImEquation26"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge&#x2013;Higgs grand unified theory (GHGUT) is proposed in the 5D RS warped space in Ref. [<xref ref-type="bibr" rid="B32">32</xref>]. The EW Higgs boson is identified with part of the fifth-dimensional component of the <inline-formula><tex-math notation="LaTeX" id="ImEquation27"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge bosons. The <inline-formula><tex-math notation="LaTeX" id="ImEquation28"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge symmetry is reduced first by two different orbifold boundary conditions (BCs) on the ultraviolet (UV) and infrared (IR) branes in the RS space. It is reduced to <inline-formula><tex-math notation="LaTeX" id="ImEquation29"><![CDATA[$SO(10)$]]></tex-math></inline-formula> by the BC on the UV brane, and to <inline-formula><tex-math notation="LaTeX" id="ImEquation30"><![CDATA[$SO(4) \times SO(7)$]]></tex-math></inline-formula> by the BC on the IR brane, the resultant symmetry being <inline-formula><tex-math notation="LaTeX" id="ImEquation31"><![CDATA[$SO(4) \times SO(6)$]]></tex-math></inline-formula>. Secondly, the brane scalar on the UV brane, which is an <inline-formula><tex-math notation="LaTeX" id="ImEquation32"><![CDATA[$SO(10)$]]></tex-math></inline-formula> spinor <inline-formula><tex-math notation="LaTeX" id="ImEquation33"><![CDATA[${\bf 16}$]]></tex-math></inline-formula>, spontaneously breaks <inline-formula><tex-math notation="LaTeX" id="ImEquation34"><![CDATA[$SO(10)$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation35"><![CDATA[$SU(5)$]]></tex-math></inline-formula> by the Higgs mechanism on the UV brane. As a net result the <inline-formula><tex-math notation="LaTeX" id="ImEquation36"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge symmetry is reduced to the SM gauge symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation37"><![CDATA[$G_{\rm SM}$]]></tex-math></inline-formula>, which is dynamically broken to <inline-formula><tex-math notation="LaTeX" id="ImEquation38"><![CDATA[$SU(3)_C \times U(1)_{\rm EM}$]]></tex-math></inline-formula> by the Hosotani mechanism.</p>
<p>Zero modes of 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation39"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation40"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> fermions in the bulk are identified with each generation of quarks and leptons. In Ref. [<xref ref-type="bibr" rid="B35">35</xref>] it is found that the observed mass spectra of the quarks and leptons can be reproduced, while additional exotic particles appear having unacceptably small masses.</p>
<p>Recently, to avoid these light exotic particles, <inline-formula><tex-math notation="LaTeX" id="ImEquation41"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT in six-dimensional (6D) hybrid warped space has been proposed in Ref. [<xref ref-type="bibr" rid="B38">38</xref>]. For neutrinos a new seesaw mechanism in 6D hybrid warped space has been formulated by using a 5D symplectic Majorana fermion [<xref ref-type="bibr" rid="B39">39</xref>], which generalizes the well-known 4D seesaw mechanism [<xref ref-type="bibr" rid="B40">40</xref>].</p>
<p>This paper is organized as follows. In <xref ref-type="sec" rid="SEC2">Sect. 2</xref>, 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation42"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT is introduced. The matter content is specified and the action is given that contains both 6D bulk and 5D brane terms. In <xref ref-type="sec" rid="SEC3">Sect. 3</xref>, a summary is given for the mass spectrum of the 4D gauge and scalar bosons originating from the 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation43"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge bosons. In <xref ref-type="sec" rid="SEC4">Sect. 4</xref>, we derive the mass spectrum of 4D SM fermions. By using these mass spectra, we evaluate the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation44"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula> in <xref ref-type="sec" rid="SEC5">Sect. 5</xref> to show that the dynamical EW symmetry breaking takes place and the 4D Higgs boson mass <inline-formula><tex-math notation="LaTeX" id="ImEquation45"><![CDATA[$m_H = 125.1\,$]]></tex-math></inline-formula>GeV is obtained. <xref ref-type="sec" rid="SEC6">Section 6</xref> is devoted to a summary and discussions. In the Appendix, the basics of the KK expansion in 6D warped space are explained.</p>
</sec>
<sec id="SEC2"><title>2. Six-dimensional gauge&#x2013;Higgs grand unification theory</title>
<p>We construct an <inline-formula><tex-math notation="LaTeX" id="ImEquation46"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge-Higgs grand unified model on the six-dimensional hybrid warped space introduced in Ref. [<xref ref-type="bibr" rid="B38">38</xref>]. The metric of generalized RS space [<xref ref-type="bibr" rid="B38">38</xref>,<xref ref-type="bibr" rid="B41">41</xref>] is given by
<disp-formula id="ptx175-M2-1"><label>(2.1)</label><tex-math notation="LaTeX" id="Equation1"><![CDATA[
\begin{align}
ds^2=e^{-2\sigma(y)}
\left(\eta_{\mu\nu}dx^\mu dx^\nu+dv^2\right)+dy^2,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation47"><![CDATA[$e^{-2\sigma(y)}$]]></tex-math></inline-formula> is a warped factor and <inline-formula><tex-math notation="LaTeX" id="ImEquation48"><![CDATA[$\eta_{\mu\nu}=\mbox{diag}(-1,+1,+1,+1)$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation49"><![CDATA[$\sigma(y)$]]></tex-math></inline-formula> satisfies <inline-formula><tex-math notation="LaTeX" id="ImEquation50"><![CDATA[$\sigma(y)=\sigma(-y)=\sigma(y+2L_5)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation51"><![CDATA[$\sigma(y)=k|y|\ \mbox{for}\ |y|\leq L_5$]]></tex-math></inline-formula>. The fifth dimension with the coordinate <inline-formula><tex-math notation="LaTeX" id="ImEquation52"><![CDATA[$y$]]></tex-math></inline-formula> behaves as the EW dimension, whereas the sixth dimension with the coordinate <inline-formula><tex-math notation="LaTeX" id="ImEquation53"><![CDATA[$v$]]></tex-math></inline-formula> is <inline-formula><tex-math notation="LaTeX" id="ImEquation54"><![CDATA[$S^1$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation55"><![CDATA[$(v\sim v+2\pi R_6)$]]></tex-math></inline-formula> and behaves as the GUT dimension. Two spacetime points <inline-formula><tex-math notation="LaTeX" id="ImEquation56"><![CDATA[$(x^\mu, y , v)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation57"><![CDATA[$(x^\mu, -y, -v)$]]></tex-math></inline-formula> are identified by the <inline-formula><tex-math notation="LaTeX" id="ImEquation58"><![CDATA[$\mathbb{Z}_2$]]></tex-math></inline-formula> transformation. As a result, the spacetime has the same topology as the orbifold <inline-formula><tex-math notation="LaTeX" id="ImEquation59"><![CDATA[$M^4\times T^2/\mathbb{Z}_2$]]></tex-math></inline-formula>. The spacetime in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-1">2.1</xref>) solves the Einstein equations with brane tensions at <inline-formula><tex-math notation="LaTeX" id="ImEquation60"><![CDATA[$y=0$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation61"><![CDATA[$y=L_5$]]></tex-math></inline-formula>. The bulk region is the anti-de-Sitter space with a negative cosmological constant <inline-formula><tex-math notation="LaTeX" id="ImEquation62"><![CDATA[$\Lambda=-10k^2$]]></tex-math></inline-formula>. The five-dimensional branes at <inline-formula><tex-math notation="LaTeX" id="ImEquation63"><![CDATA[$y=0$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation64"><![CDATA[$y=L_5$]]></tex-math></inline-formula> have the same topology as <inline-formula><tex-math notation="LaTeX" id="ImEquation65"><![CDATA[$M^4 \times S^1$]]></tex-math></inline-formula>.</p>
<p>The extra-dimensional space has four fixed points under <inline-formula><tex-math notation="LaTeX" id="ImEquation66"><![CDATA[$\mathbb{Z}_2$]]></tex-math></inline-formula>: <inline-formula><tex-math notation="LaTeX" id="ImEquation67"><![CDATA[$(y_0,v_0)=(0,0)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation68"><![CDATA[$(y_1,v_1)=(L_5,0)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation69"><![CDATA[$(y_2,v_2)=(0,\pi R_6)$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation70"><![CDATA[$(y_3,v_3)=(L_5,\pi R_6)$]]></tex-math></inline-formula>. In terms of the conformal coordinate <inline-formula><tex-math notation="LaTeX" id="ImEquation71"><![CDATA[$z=e^{ky} \ (1\leq z\leq z_L=e^{kL_5})$]]></tex-math></inline-formula> in the region <inline-formula><tex-math notation="LaTeX" id="ImEquation72"><![CDATA[$0\leq y \leq L_5$]]></tex-math></inline-formula>, the metric becomes
<disp-formula id="ptx175-M2-2"><label>(2.2)</label><tex-math notation="LaTeX" id="Equation2"><![CDATA[
\begin{align}
ds^2=\frac{1}{z^2}\left(
\eta_{\mu\nu}dx^\mu dx^\nu+dv^2+\frac{dz^2}{k^2}\right)\!.
\end{align}]]></tex-math></disp-formula></p>
<p>The fifth- and sixth-dimensional KK mass scales are given by <inline-formula><tex-math notation="LaTeX" id="ImEquation73"><![CDATA[$m_{\rm KK_5}=\pi k/(e^{kL_5}-1)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation74"><![CDATA[$m_{\rm KK_6}=R_6^{-1}$]]></tex-math></inline-formula>. The warp factor is supposed to be large; <inline-formula><tex-math notation="LaTeX" id="ImEquation75"><![CDATA[$z_L = e^{kL_5} \gg 1$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation76"><![CDATA[$m_{\rm KK_6}$]]></tex-math></inline-formula> is expected to be a GUT scale while <inline-formula><tex-math notation="LaTeX" id="ImEquation77"><![CDATA[$m_{\rm KK_5}$]]></tex-math></inline-formula> is <inline-formula><tex-math notation="LaTeX" id="ImEquation78"><![CDATA[$O(10) \,$]]></tex-math></inline-formula>TeV so that <inline-formula><tex-math notation="LaTeX" id="ImEquation79"><![CDATA[$m_{\rm KK_6}\gg m_{\rm KK_5}$]]></tex-math></inline-formula>.</p>
<p>Parity transformations <inline-formula><tex-math notation="LaTeX" id="ImEquation80"><![CDATA[$P_j$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation81"><![CDATA[$(j=0,1,2,3)$]]></tex-math></inline-formula> around the four fixed points are defined as <inline-formula><tex-math notation="LaTeX" id="ImEquation82"><![CDATA[$(x^\mu,y_j+y, v_j+v) \rightarrow (x^\mu,y_j-y,v_j-v)$]]></tex-math></inline-formula>. Only three of the four parity transformations <inline-formula><tex-math notation="LaTeX" id="ImEquation83"><![CDATA[$P_j$]]></tex-math></inline-formula> are independent. They satisfy the relation <inline-formula><tex-math notation="LaTeX" id="ImEquation84"><![CDATA[$P_3=P_2P_0P_1=P_1P_0P_2$]]></tex-math></inline-formula>.</p>
<p>We adopt orbifold BCs such that <inline-formula><tex-math notation="LaTeX" id="ImEquation85"><![CDATA[$P_{0}=P_{1}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation86"><![CDATA[$P_{2}=P_{3}$]]></tex-math></inline-formula>, which enables us to avoid the problem of unwanted light exotic fermions mentioned in the Introduction. More specifically, we choose BCs such that <inline-formula><tex-math notation="LaTeX" id="ImEquation87"><![CDATA[$P_{0}=P_{1}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation88"><![CDATA[$P_{2}=P_{3}$]]></tex-math></inline-formula> break <inline-formula><tex-math notation="LaTeX" id="ImEquation89"><![CDATA[$SO(11)$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation90"><![CDATA[$SO(4)\times SO(7)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation91"><![CDATA[$SO(10)$]]></tex-math></inline-formula>, respectively. The orbifold BCs reduce <inline-formula><tex-math notation="LaTeX" id="ImEquation92"><![CDATA[$SO(11)$]]></tex-math></inline-formula> symmetry to the Pati&#x2013;Salam symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation93"><![CDATA[$SO(4) \times SO(6) \simeq SU(2)_L\times SU(2)_R\times SU(4)_C=:G_{\rm PS}$]]></tex-math></inline-formula>. We note that the fifth- and sixth-dimensional loop translations <inline-formula><tex-math notation="LaTeX" id="ImEquation94"><![CDATA[$U_5:(x^\mu,y,v)\to(x^\mu,y+2L_5,v)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation95"><![CDATA[$U_6:(x^\mu,y,v)\to(x^\mu,y,v+2\pi R_6)$]]></tex-math></inline-formula> are related to the <inline-formula><tex-math notation="LaTeX" id="ImEquation96"><![CDATA[$P_j$]]></tex-math></inline-formula>s by <inline-formula><tex-math notation="LaTeX" id="ImEquation97"><![CDATA[$U_5=P_1P_0=P_3P_2$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation98"><![CDATA[$U_6=P_2P_0=P_3P_1$]]></tex-math></inline-formula>.</p>
<sec id="SEC2.1"><title>2.1. 6D bulk and 5D brane fields and orbifold boundary conditions</title>
<p>The matter content in the <inline-formula><tex-math notation="LaTeX" id="ImEquation99"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT consists of 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation100"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge bosons <inline-formula><tex-math notation="LaTeX" id="ImEquation101"><![CDATA[$A_M$]]></tex-math></inline-formula>, 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation102"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation103"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> Weyl fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation104"><![CDATA[$\Psi_{\bf 32}^{\alpha}(x,y,v)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation105"><![CDATA[$(\alpha=1,2,3,4)$]]></tex-math></inline-formula>, 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation106"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation107"><![CDATA[${\bf 11}$]]></tex-math></inline-formula> Dirac fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation108"><![CDATA[$\Psi_{\bf 11}^{\beta}(x,y,v)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation109"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}(x,y,v)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation110"><![CDATA[$(\beta=1,2,3)$]]></tex-math></inline-formula> in the six-dimensional bulk space, and a 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation111"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation112"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> brane scalar boson <inline-formula><tex-math notation="LaTeX" id="ImEquation113"><![CDATA[$\Phi_{\bf 32}(x,v)$]]></tex-math></inline-formula> and 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation114"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation115"><![CDATA[${\bf 1}$]]></tex-math></inline-formula> brane symplectic Majorana fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation116"><![CDATA[$\chi_{\bf 1}^\beta (x,v)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation117"><![CDATA[$(\beta=1,2,3)$]]></tex-math></inline-formula> on the UV brane <inline-formula><tex-math notation="LaTeX" id="ImEquation118"><![CDATA[$y=0$]]></tex-math></inline-formula> [<xref ref-type="bibr" rid="B38">38</xref>]. Their orbifold BCs are listed below:</p>
<p><list list-type="bullet">
<list-item><p>For the 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation119"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge boson <inline-formula><tex-math notation="LaTeX" id="ImEquation120"><![CDATA[$A_M$]]></tex-math></inline-formula>, the orbifold BCs are given by
<disp-formula id="ptx175-M2-3"><label>(2.3)</label><tex-math notation="LaTeX" id="Equation3"><![CDATA[
\begin{align}
&\left(
\begin{array}{c}
A_\mu\\
A_{y}\\
A_{v}\\
\end{array}
\right)
(x,y_j-y,v_j-v)
=P_{j}
\left(
\begin{array}{c}
A_\mu\\
-A_{y}\\
-A_{v}\\
\end{array}
\right)
(x,y_j+y,v_j+v)P_{j}^{-1},
\end{align}]]></tex-math></disp-formula>
where in the <inline-formula><tex-math notation="LaTeX" id="ImEquation121"><![CDATA[$SO(11)$]]></tex-math></inline-formula> vector representation we take the orbifold BCs <inline-formula><tex-math notation="LaTeX" id="ImEquation122"><![CDATA[$P_{0}=P_{1}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation123"><![CDATA[$P_{2}=P_{3}$]]></tex-math></inline-formula> as
<disp-formula id="ptx175-M2-4"><label>(2.4)</label><tex-math notation="LaTeX" id="Equation4"><![CDATA[
\begin{align}
P_{0}^{\rm vec}=P_{1}^{\rm vec}=\mbox{diag}(I_4,-I_7),\ \ \
P_{2}^{\rm vec}=P_{3}^{\rm vec}=\mbox{diag}(I_{10},-I_1).
\end{align}]]></tex-math></disp-formula></p>
<p>By using <inline-formula><tex-math notation="LaTeX" id="ImEquation124"><![CDATA[$(P_0=P_1,\ P_2=P_3)$]]></tex-math></inline-formula>, the parity assignment of <inline-formula><tex-math notation="LaTeX" id="ImEquation125"><![CDATA[$A_\mu$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation126"><![CDATA[$A_y$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation127"><![CDATA[$A_v$]]></tex-math></inline-formula> are summarized in <xref ref-type="table" rid="T1">Table 1</xref>.</p></list-item>
<list-item><p>For the four 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation128"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor <bold>32</bold> bulk Weyl fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation129"><![CDATA[$\Psi_{\bf 32}^{\alpha}(x,y,v)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation130"><![CDATA[$(\alpha=1,2,3,4)$]]></tex-math></inline-formula>, the orbifold BCs are given by
<disp-formula id="ptx175-M2-5"><label>(2.5)</label><tex-math notation="LaTeX" id="Equation5"><![CDATA[
\begin{align}
\Psi_{{\bf 32}}^{\alpha}(x,y_j-y,v_j-v)
=\eta_j^\alpha\overline{\gamma}P_j^{\rm sp}
\Psi_{\bf 32}^{\alpha}(x,y_j+y,v_j+v),
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation131"><![CDATA[$\eta_j^{\alpha}=\pm 1$]]></tex-math></inline-formula>. Six-dimensional Dirac matrices <inline-formula><tex-math notation="LaTeX" id="ImEquation132"><![CDATA[$\gamma^a$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation133"><![CDATA[$(a=1,2,\ldots,6)$]]></tex-math></inline-formula> satisfy <inline-formula><tex-math notation="LaTeX" id="ImEquation134"><![CDATA[$\{\gamma^a,\gamma^b\}=2\eta^{ab}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation135"><![CDATA[$(\eta^{ab}=\mbox{diag}(-I_1,I_5))$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation136"><![CDATA[$\overline{\gamma}:=-i\gamma^5\gamma^6 =\gamma_{6D}^{7}\gamma_{4D}^{5}=\gamma_{4D}^{5}\gamma_{6D}^{7}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation137"><![CDATA[$\gamma_{4D}^{5}=I_2\otimes\sigma^3\otimes I_2$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation138"><![CDATA[$\gamma_{6D}^{7}=I_4\otimes\sigma^3$]]></tex-math></inline-formula>. The <inline-formula><tex-math notation="LaTeX" id="ImEquation139"><![CDATA[$P_j^{\rm sp}$]]></tex-math></inline-formula>s are
<disp-formula id="ptx175-M2-6"><label>(2.6)</label><tex-math notation="LaTeX" id="Equation6"><![CDATA[
\begin{align}
&P_{0}^{\rm sp}=P_{1}^{\rm sp}=I_2\otimes \sigma^3\otimes I_8,\ \ \
P_{2}^{\rm sp}=P_{3}^{\rm sp}=I_{16}\otimes\sigma^3.
\end{align}]]></tex-math></disp-formula></p>
<p>To ensure the 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation140"><![CDATA[$SO(11)$]]></tex-math></inline-formula> chiral anomaly cancellation, we assign <inline-formula><tex-math notation="LaTeX" id="ImEquation141"><![CDATA[$\gamma_{6D}^7=+1$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation142"><![CDATA[$\alpha=1,2$]]></tex-math></inline-formula>; <inline-formula><tex-math notation="LaTeX" id="ImEquation143"><![CDATA[$\gamma_{6D}^7=-1$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation144"><![CDATA[$\alpha=3,4$]]></tex-math></inline-formula>. We take <inline-formula><tex-math notation="LaTeX" id="ImEquation145"><![CDATA[$\eta_j^{1,2}=-1$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation146"><![CDATA[$\eta_j^3=1$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation147"><![CDATA[$\eta_{0,2}^4=-\eta_{1,3}^4=1$]]></tex-math></inline-formula>. The parity assignment
<disp-formula id="ptx175-M2-7"><label>(2.7)</label><tex-math notation="LaTeX" id="Equation7"><![CDATA[
\begin{align}
\left(
\begin{array}{@{}cc@{}}
\eta_2^\alpha \overline{\gamma}P_2^{\rm sp}&
\eta_3^\alpha \overline{\gamma}P_3^{\rm sp}\\[6pt]
\eta_0^\alpha \overline{\gamma}P_0^{\rm sp}&
\eta_1^\alpha \overline{\gamma}P_1^{\rm sp}\\
\end{array}
\right)
\end{align}]]></tex-math></disp-formula>
of the 4D left- and right-handed components of <inline-formula><tex-math notation="LaTeX" id="ImEquation148"><![CDATA[$\Psi_{\bf 32}^{\alpha}$]]></tex-math></inline-formula> are summarized in <xref ref-type="table" rid="T2">Table 2</xref>. We find that <inline-formula><tex-math notation="LaTeX" id="ImEquation149"><![CDATA[$\Psi_{\bf 32}^{\alpha}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation150"><![CDATA[$(\alpha=1,2,3)$]]></tex-math></inline-formula> has zero modes, corresponding to one generation of quarks and leptons for each <inline-formula><tex-math notation="LaTeX" id="ImEquation151"><![CDATA[$\alpha$]]></tex-math></inline-formula>. The corresponding names adopted in Ref. [<xref ref-type="bibr" rid="B35">35</xref>] are also listed in <xref ref-type="table" rid="T2">Table 2</xref> for <inline-formula><tex-math notation="LaTeX" id="ImEquation152"><![CDATA[$\alpha=1,2,3$]]></tex-math></inline-formula>.</p></list-item>
<list-item><p>For the three 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation153"><![CDATA[$SO(11)$]]></tex-math></inline-formula> vector <bold>11</bold> bulk Dirac fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation154"><![CDATA[$\Psi_{\bf 11}^{\beta}(x,y,v)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation155"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}(x,y,v)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation156"><![CDATA[$(\beta=1,2,3)$]]></tex-math></inline-formula>, the orbifold BCs are given by
<disp-formula id="ptx175-M2-8"><label>(2.8)</label><tex-math notation="LaTeX" id="Equation8"><![CDATA[
\begin{align}
\Psi_{{\bf 11}}^{(\prime)\beta}(x,y_j-y,v_j-v)
=\eta_j^{(\prime)\beta}\overline{\gamma}P_j^{\rm vec}
\Psi_{\bf 11}^{(\prime)\beta}(x,y_j+y,v_j+v),
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation157"><![CDATA[$\eta_{0,1}^{\beta}=-\eta_{2,3}^{\beta}=-1$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation158"><![CDATA[$\Psi_{\bf 11}^{\beta}$]]></tex-math></inline-formula>; <inline-formula><tex-math notation="LaTeX" id="ImEquation159"><![CDATA[$\eta_{0,1}^{\prime\beta}=\eta_{2,3}^{\prime\beta}=-1$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation160"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula>. The parity assignments <inline-formula><tex-math notation="LaTeX" id="ImEquation161"><![CDATA[$(\eta_{0}^{\prime\beta}\overline{\gamma}P_0^{\rm vec}= \eta_{1}^{\prime\beta}\overline{\gamma}P_1^{\rm vec},\ \eta_{2}^{\prime\beta}\overline{\gamma}P_2^{\rm vec}= \eta_{3}^{\prime\beta}\overline{\gamma}P_3^{\rm vec})$]]></tex-math></inline-formula> of the 4D left- and right-handed components of <inline-formula><tex-math notation="LaTeX" id="ImEquation162"><![CDATA[$\Psi_{\bf 11}^{(\prime)\beta}$]]></tex-math></inline-formula> are summarized in <xref ref-type="table" rid="T3">Table 3</xref>.</p></list-item>
<list-item><p>For the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation163"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor <inline-formula><tex-math notation="LaTeX" id="ImEquation164"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> brane scalar field on the UV brane at <inline-formula><tex-math notation="LaTeX" id="ImEquation165"><![CDATA[$y=0$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation166"><![CDATA[$\Phi_{\bf 32}(x,v)$]]></tex-math></inline-formula>, the orbifold BCs are given by
<disp-formula id="ptx175-M2-9"><label>(2.9)</label><tex-math notation="LaTeX" id="Equation9"><![CDATA[
\begin{align}
\Phi_{\bf 32}(x,v_j-v)=\eta_j P_j^{\rm sp}\Phi_{\bf 32}(x,v_j+v),
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation167"><![CDATA[$j=0,2$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation168"><![CDATA[$\eta_0=-\eta_2=-1$]]></tex-math></inline-formula>. The components of <inline-formula><tex-math notation="LaTeX" id="ImEquation169"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation170"><![CDATA[$({\bf 1,2,\overline{4}})$]]></tex-math></inline-formula> have zero modes, one of which corresponds to the <inline-formula><tex-math notation="LaTeX" id="ImEquation171"><![CDATA[$SU(5)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation172"><![CDATA[${\bf 1}$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation173"><![CDATA[$SO(10)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation174"><![CDATA[${\bf 16}$]]></tex-math></inline-formula>. It is responsible for reducing <inline-formula><tex-math notation="LaTeX" id="ImEquation175"><![CDATA[$SO(11)$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation176"><![CDATA[$SU(5)$]]></tex-math></inline-formula> at the UV brane <inline-formula><tex-math notation="LaTeX" id="ImEquation177"><![CDATA[$y=0$]]></tex-math></inline-formula>. The BCs of <inline-formula><tex-math notation="LaTeX" id="ImEquation178"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> are summarized in <xref ref-type="table" rid="T4">Table 4</xref>.</p></list-item>
<list-item><p>For the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation179"><![CDATA[$SO(11)$]]></tex-math></inline-formula> singlet fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation180"><![CDATA[$\chi^\beta (x,v) = \chi_{\bf 1}^\beta (x,v)$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation181"><![CDATA[$\beta =1,2,3$]]></tex-math></inline-formula>) at <inline-formula><tex-math notation="LaTeX" id="ImEquation182"><![CDATA[$y=0$]]></tex-math></inline-formula>, their orbifold BCs are given by
<disp-formula id="ptx175-M2-10"><label>(2.10)</label><tex-math notation="LaTeX" id="Equation10"><![CDATA[
\begin{align}
\chi^\beta (x,v_j-v)=\overline{\gamma} \chi^\beta (x,v_j+v)\ \
(\,j=0,2).
\end{align}]]></tex-math></disp-formula></p>
<p>These brane fields also satisfy the 5D symplectic Majorana condition <inline-formula><tex-math notation="LaTeX" id="ImEquation183"><![CDATA[$\chi^C=\widetilde{\chi}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation184"><![CDATA[$\widetilde{\chi}:=i\overline{\Gamma}\chi$]]></tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX" id="ImEquation185"><![CDATA[$\overline{\Gamma}:=\gamma_{4D}^5\gamma^6$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M2-11"><label>(2.11)</label><tex-math notation="LaTeX" id="Equation11"><![CDATA[
\begin{align}
\chi=
\left(
\begin{array}{c}
\xi_+\\
\eta_+\\
\xi_-\\
\eta_-\\
\end{array}
\right)\!,\ \ \
\chi^C=
\left(
\begin{array}{c}
+\eta_+^C\\
-\xi_+^C\\
-\eta_-^C\\
+\xi_-^C\\
\end{array}
\right)
=e^{i\delta_C}
\left(
\begin{array}{c}
-\sigma^2\eta_+^*\\
-\sigma^2\xi_+^*\\
+\sigma^2\eta_-^*\\
+\sigma^2\xi_-^*\\
\end{array}
\right)
=\left(
\begin{array}{c}
+\xi_-\\
-\eta_-\\
-\xi_+\\
+\eta_+\\
\end{array}
\right)=
\widetilde{\chi}.
\end{align}]]></tex-math></disp-formula></p>
<p>Here the generation index <inline-formula><tex-math notation="LaTeX" id="ImEquation186"><![CDATA[$\beta$]]></tex-math></inline-formula> has been suppressed.</p></list-item>
</list></p>
<p><table-wrap id="T1" orientation="portrait" position="float"><label>Table 1.</label><caption><p>Parity assignment <inline-formula><tex-math notation="LaTeX" id="ImEquation187"><![CDATA[$(P_0, P_2)=(P_1, P_3)$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation188"><![CDATA[$A_\mu$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation189"><![CDATA[$A_y$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation190"><![CDATA[$A_v$]]></tex-math></inline-formula> in <inline-formula><tex-math notation="LaTeX" id="ImEquation191"><![CDATA[$G_{\rm PS}=SU(2)_L\times SU(2)_R\times SU(4)_C$]]></tex-math></inline-formula>.</p></caption>
<table frame="hsides" rules="groups">
<thead align="left">
<tr>
<th align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation192"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation193"><![CDATA[$A_\mu$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation194"><![CDATA[$A_y$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation195"><![CDATA[$A_v$]]></tex-math></inline-formula></th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">(<bold>3,1,1</bold>)</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation196"><![CDATA[$(+,+)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation197"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation198"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left">(<bold>1,3,1</bold>)</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation199"><![CDATA[$(+,+)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation200"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation201"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left">(<bold>1,1,15</bold>)</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation202"><![CDATA[$(+,+)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation203"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation204"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left">(<bold>2,2,6</bold>)</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation205"><![CDATA[$(-,+)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation206"><![CDATA[$(+,-)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation207"><![CDATA[$(+,-)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left">(<bold>2,2,1</bold>)</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation208"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation209"><![CDATA[$(+,+)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation210"><![CDATA[$(+,+)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left">(<bold>1,1,6</bold>)</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation211"><![CDATA[$(+,-)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation212"><![CDATA[$(-,+)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation213"><![CDATA[$(-,+)$]]></tex-math></inline-formula></td>
</tr>
</tbody>
</table>
</table-wrap></p>
<p><table-wrap id="T2" orientation="portrait" position="float"><label>Table 2.</label><caption><p>Parity assignment <inline-formula><tex-math notation="LaTeX" id="ImEquation214"><![CDATA[$\left( {\matrix{ {{P_2}} \hfill & {{P_3}} \hfill \cr {{P_0}} \hfill & {{P_1}} \hfill \cr } } \right)$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-7">2.7</xref>) of <inline-formula><tex-math notation="LaTeX" id="ImEquation215"><![CDATA[$\Psi_{\bf 32}^{\alpha}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation216"><![CDATA[$(\alpha=1,2,3,4)$]]></tex-math></inline-formula> in <inline-formula><tex-math notation="LaTeX" id="ImEquation217"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula>.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" orientation="portrait" position="float" xlink:href="ptx175TF2.tif"/>
</table-wrap></p>
<p><table-wrap id="T3" orientation="portrait" position="float"><label>Table 3.</label><caption><p>Parity assignment <inline-formula><tex-math notation="LaTeX" id="ImEquation218"><![CDATA[$(P_0, P_2)=(P_1, P_3)$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation219"><![CDATA[$\Psi_{\bf 11}^{(\prime)\beta}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation220"><![CDATA[$(\beta=1,2,3)$]]></tex-math></inline-formula> in <inline-formula><tex-math notation="LaTeX" id="ImEquation221"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula>.</p></caption>
<table frame="hsides" rules="groups">
<thead align="left">
<tr>
<th align="center" colspan="6"><inline-formula><tex-math notation="LaTeX" id="ImEquation222"><![CDATA[$\Psi_{\bf 11}^{\beta}$]]></tex-math></inline-formula></th>
</tr>
<tr>
<th align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation223"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula></th>
<th align="center">Left<inline-formula><tex-math notation="LaTeX" id="ImEquation224"><![CDATA[${}_+$]]></tex-math></inline-formula></th>
<th align="center">Right<inline-formula><tex-math notation="LaTeX" id="ImEquation225"><![CDATA[${}_+$]]></tex-math></inline-formula></th>
<th align="center">Left<inline-formula><tex-math notation="LaTeX" id="ImEquation226"><![CDATA[${}_-$]]></tex-math></inline-formula></th>
<th align="center">Right<inline-formula><tex-math notation="LaTeX" id="ImEquation227"><![CDATA[${}_-$]]></tex-math></inline-formula></th>
<th align="center">name</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation228"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation229"><![CDATA[$\left(+,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation230"><![CDATA[$\left(-,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation231"><![CDATA[$\left(-,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation232"><![CDATA[$\left(+,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation233"><![CDATA[\begin{matrix} N & \hat E \cr E & \hat N \end{matrix}]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation234"><![CDATA[$({\bf 1,1,6})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation235"><![CDATA[$\left(-,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation236"><![CDATA[$\left(+,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation237"><![CDATA[$\left(+,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation238"><![CDATA[$\left(-,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation239"><![CDATA[$D_j, \hat D_j$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation240"><![CDATA[$({\bf 1,1,1})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation241"><![CDATA[$\left(-,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation242"><![CDATA[$\left(+,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation243"><![CDATA[$\left(+,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation244"><![CDATA[$\left(-,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation245"><![CDATA[$S$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="center" colspan="6"><inline-formula><tex-math notation="LaTeX" id="ImEquation246"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation247"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula></td>
<td align="center">Left<inline-formula><tex-math notation="LaTeX" id="ImEquation248"><![CDATA[${}_+$]]></tex-math></inline-formula></td>
<td align="center">Right<inline-formula><tex-math notation="LaTeX" id="ImEquation249"><![CDATA[${}_+$]]></tex-math></inline-formula></td>
<td align="center">Left<inline-formula><tex-math notation="LaTeX" id="ImEquation250"><![CDATA[${}_-$]]></tex-math></inline-formula></td>
<td align="center">Right<inline-formula><tex-math notation="LaTeX" id="ImEquation251"><![CDATA[${}_-$]]></tex-math></inline-formula></td>
<td align="center">name</td>
</tr>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation252"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation253"><![CDATA[$\left(+,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation254"><![CDATA[$\left(-,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation255"><![CDATA[$\left(-,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation256"><![CDATA[$\left(+,+\right)$]]></tex-math></inline-formula>
</td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation257"><![CDATA[\begin{matrix} N' & \hat E' \cr E' & \hat N' \end{matrix}]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation258"><![CDATA[$({\bf 1,1,6})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation259"><![CDATA[$\left(-,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation260"><![CDATA[$\left(+,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation261"><![CDATA[$\left(+,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation262"><![CDATA[$\left(-,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation263"><![CDATA[$D_j' , \hat D_j'$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation264"><![CDATA[$({\bf 1,1,1})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation265"><![CDATA[$\left(-,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation266"><![CDATA[$\left(+,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation267"><![CDATA[$\left(+,+\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation268"><![CDATA[$\left(-,-\right)$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation269"><![CDATA[$S'$]]></tex-math></inline-formula></td>
</tr>
</tbody>
</table>
</table-wrap></p>
<p><table-wrap id="T4" orientation="portrait" position="float"><label>Table 4.</label><caption><p>Parity assignment <inline-formula><tex-math notation="LaTeX" id="ImEquation270"><![CDATA[$(P_0, P_2)$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation271"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula>
in <inline-formula><tex-math notation="LaTeX" id="ImEquation272"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula>.</p></caption>
<table frame="hsides" rules="groups">
<thead align="left">
<tr>
<th align="center" colspan="2"><inline-formula><tex-math notation="LaTeX" id="ImEquation273"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula></th>
</tr>
<tr>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation274"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula></th>
<th align="center">BCs</th>
</tr>
</thead>
<tbody>
<tr>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation275"><![CDATA[$({\bf 2,1,4})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation276"><![CDATA[$(-,+)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation277"><![CDATA[$({\bf 1,2,\overline{4}})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation278"><![CDATA[$(+,+)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation279"><![CDATA[$({\bf 2,1,\overline{4}})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation280"><![CDATA[$(-,-)$]]></tex-math></inline-formula></td>
</tr>
<tr>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation281"><![CDATA[$({\bf 1,2,4})$]]></tex-math></inline-formula></td>
<td align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation282"><![CDATA[$(+,-)$]]></tex-math></inline-formula></td>
</tr>
</tbody>
</table>
</table-wrap></p>
</sec>
<sec id="SEC2.2"><title>2.2. Action</title>
<p>The action consists of the 6D bulk and 5D brane terms.</p>
<sec id="SEC2.2.1"><title>2.2.1. Bulk terms</title>
<p>The bulk part of the action is given by
<disp-formula id="ptx175-M2-12"><label>(2.12)</label><tex-math notation="LaTeX" id="Equation12"><![CDATA[
\begin{align}
S_{\rm bulk}= S_{\rm bulk}^{\rm gauge}+S_{\rm bulk}^{\rm fermion},
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation283"><![CDATA[$S_{\rm bulk}^{\rm gauge}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation284"><![CDATA[$S_{\rm bulk}^{\rm fermion}$]]></tex-math></inline-formula> are bulk actions of gauge and fermion fields, respectively. The action of <inline-formula><tex-math notation="LaTeX" id="ImEquation285"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge field <bold>55</bold> <inline-formula><tex-math notation="LaTeX" id="ImEquation286"><![CDATA[$A_M(x,y,v)$]]></tex-math></inline-formula> is
<disp-formula id="ptx175-M2-13"><label>(2.13)</label><tex-math notation="LaTeX" id="Equation13"><![CDATA[
\begin{align}
S_{\rm bulk}^{\rm gauge}=&
\int d^6x\sqrt{-\mbox{det}G}\bigg[-\mbox{tr}\left(
\frac{1}{4}F_{}^{MN}F_{MN}
+\frac{1}{2\xi}(f_{\rm gf})^2+{\cal L}_{\rm gh}\right)\bigg],
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation287"><![CDATA[$\sqrt{-\mbox{det}G}=1/k z^6$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation288"><![CDATA[$z=e^{ky}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation289"><![CDATA[$M,N=0,1,2,3,5,6$]]></tex-math></inline-formula>, and the field strength <inline-formula><tex-math notation="LaTeX" id="ImEquation290"><![CDATA[$F_{MN}$]]></tex-math></inline-formula> is defined by
<disp-formula id="ptx175-M2-14"><label>(2.14)</label><tex-math notation="LaTeX" id="Equation14"><![CDATA[
\begin{align}
F_{MN}&:=
\partial_MA_N-\partial_NA_M-i g[A_M,A_N].
\end{align}]]></tex-math></disp-formula></p>
<p>We take the following gauge-fixing and ghost terms:
<disp-formula id="ptx175-M2-15"><label>(2.15)</label><tex-math notation="LaTeX" id="Equation15"><![CDATA[
\begin{align}
f_{\rm gf}&=
z^2\left\{\eta^{\mu\nu}{\cal D}_\mu^c A_\nu^q
+{\cal D}_6^c A_6^q
+\xi k^2z^2{\cal D}_z^c\left(\frac{A_z^q}{z^2}\right)\right\}\!,\cr
{\cal L}_{\rm gh}&=
\bar{c}\left\{
\eta^{\mu\nu}{\cal D}_\mu^c{\cal D}_\nu^{c+q}
+{\cal D}_6^c{\cal D}_6^{c+q}
+\xi k^2z^2{\cal D}_z^c\frac{1}{z^2}{\cal D}_z^{c+q}\right\}c,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation291"><![CDATA[$\mu,\nu=0,1,2,3$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation292"><![CDATA[$\eta^{\mu\nu}=\eta_{\mu\nu}=\mbox{diag}(-1,1,1,1)$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation293"><![CDATA[$A_M=A_M^c+A_M^q$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation294"><![CDATA[${\cal D}_M^{c}B=\partial_M B-ig[A_M^c,B]$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation295"><![CDATA[${\cal D}_M^{c+q}B=\partial_M B-ig[A_M,B]$]]></tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX" id="ImEquation296"><![CDATA[$B=A_\mu^q,A_z^q/z^2,A_6^q$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation297"><![CDATA[$c$]]></tex-math></inline-formula>.</p>
<p>The action of fermions in <inline-formula><tex-math notation="LaTeX" id="ImEquation298"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor and vector representations <bold>32</bold> and <bold>11</bold>, <inline-formula><tex-math notation="LaTeX" id="ImEquation299"><![CDATA[$\Psi_{\bf 32}^{\alpha}(x,y,v)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation300"><![CDATA[$\Psi_{\bf 11}^{\beta}(x,y,v)$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation301"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}(x,y,v)$]]></tex-math></inline-formula>, is given by
<disp-formula id="ptx175-M2-16"><label>(2.16)</label><tex-math notation="LaTeX" id="Equation16"><![CDATA[
\begin{align}
S_{\rm bulk}^{\rm fermion}=&
\int d^6x\sqrt{-\mbox{det}G} \,
\left\{
\sum_{\alpha=1}^4\overline{\Psi_{\bf 32}^{\alpha}}{\cal D}
(c_{\Psi_{\bf 32}^{\alpha}})
\Psi_{\bf 32}^{\alpha}\right. \cr
&\hspace{3.7cm} \left.+\sum_{\beta=1}^3
\overline{\Psi_{\bf 11}^{\beta}}{\cal D}
(c_{\Psi_{\bf 11}^{\beta}})
\Psi_{\bf 11}^{\beta}
+\sum_{\beta=1}^3
\overline{\Psi_{\bf 11}^{\prime\beta}}{\cal D}
(c_{\Psi_{\bf 11}^{\prime\beta}})
\Psi_{\bf 11}^{\prime\beta}
\right\}\!,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation302"><![CDATA[$\overline{\Psi_{\bf 32}^{\alpha}}:= i{\Psi_{\bf 32}^{\alpha\dagger}}\gamma^0$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation303"><![CDATA[$c_{\Psi_{\bf 32}^{\alpha}}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation304"><![CDATA[$c_{\Psi_{\bf 11}^{\beta}}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation305"><![CDATA[$c_{\Psi_{\bf 11}^{\prime\beta}}$]]></tex-math></inline-formula> are bulk mass parameters, and <inline-formula><tex-math notation="LaTeX" id="ImEquation306"><![CDATA[${\cal D}(c)$]]></tex-math></inline-formula> is a covariant derivative given by
<disp-formula id="ptx175-M2-17"><label>(2.17)</label><tex-math notation="LaTeX" id="Equation17"><![CDATA[
\begin{align}
{\cal D}(c)=z\left\{
\gamma^\mu D_\mu+\gamma^6 D_v-\frac{5}{2}\frac{\sigma'}{z}\gamma^5
+\sigma'\gamma^5 D_z+ic\frac{\sigma'}{z}\gamma^6\right\}\!,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation307"><![CDATA[$\sigma'(y):=d\sigma(y)/dy$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation308"><![CDATA[$\sigma'(y) =k$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation309"><![CDATA[$0< y < L_5$]]></tex-math></inline-formula>.</p>
<p>Here, we introduce <inline-formula><tex-math notation="LaTeX" id="ImEquation310"><![CDATA[$\check{\Psi}$]]></tex-math></inline-formula> defined by
<disp-formula id="ptx175-M2-18"><label>(2.18)</label><tex-math notation="LaTeX" id="Equation18"><![CDATA[
\begin{align}
\check{\Psi}:=\frac{1}{z^{5/2}}\Psi,\ \
\left(D_z-\frac{5}{2}\frac{1}{z}\right)\Psi
=z^{5/2}D_z\check{\Psi},
\end{align}]]></tex-math></disp-formula>
which turns out to be convenient for discussing mass spectra for fermions in <xref ref-type="sec" rid="SEC4">Sect. 4</xref>. The bulk part of the bulk fermion action becomes
<disp-formula id="ptx175-M2-19"><label>(2.19)</label><tex-math notation="LaTeX" id="Equation19"><![CDATA[
\begin{align}
S=&\int d^4x\int_0^{2\pi R_6}dv\int_1^{z_L}\frac{dz}{k}
\Bigg[
\overline{\check{\Psi}_{\bf 32}^{\alpha}}
\left(\gamma^\mu D_\mu+\gamma^6 D_v
+\sigma'\gamma^5 D_z+ic_{\Psi_{\bf 32}}\frac{\sigma'}{z}\gamma^6\right)
\check{\Psi}_{\bf 32}^{\alpha}\nonumber\\
&\hspace{10.5em}
+\overline{\check{\Psi}_{\bf 11}^{\beta}}
\left(\gamma^\mu D_\mu+\gamma^6 D_v
+\sigma'\gamma^5 D_z+ic_{\Psi_{\bf 11}}\frac{\sigma'}{z}\gamma^6\right)
\check{\Psi}_{\bf 11}^{\beta}
\nonumber\\
&\hspace{10.5em}
+\overline{\check{\Psi}_{\bf 11}^{\prime\beta}}
\left(\gamma^\mu D_\mu+\gamma^6 D_v
+\sigma'\gamma^5 D_z+ic_{\Psi_{\bf 11}'}\frac{\sigma'}{z}\gamma^6\right)
\check{\Psi}_{\bf 11}^{\prime\beta}
\Bigg].
\end{align}]]></tex-math></disp-formula></p>
</sec>
<sec id="SEC2.2.2"><title>2.2.2. Action for the brane scalar <inline-formula><tex-math notation="LaTeX" id="ImEquation311"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> and the Higgs mechanism</title>
<p>We consider the 5D brane terms for a brane scalar <inline-formula><tex-math notation="LaTeX" id="ImEquation312"><![CDATA[$\Phi_{\bf 32}(x,v)$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M2-20"><label>(2.20)</label><tex-math notation="LaTeX" id="Equation20"><![CDATA[
\begin{align}
&S_{\rm 5D\ brane}^{\rm brane}=
\int d^6x\sqrt{-\mbox{det}G} \, \delta(y)
{\cal L}_{\Phi_{\bf 32}} , \cr
&{\cal L}_{\Phi_{\bf 32}}:=
-(D_\mu\Phi_{\bf 32})^{\dagger}D^\mu\Phi_{\bf 32}
-(D_v\Phi_{\bf 32})^{\dagger}D^v\Phi_{\bf 32}
-\lambda_{\Phi_{\bf 32}}
\big( \Phi_{\bf 32}^\dagger\Phi_{\bf 32}-|w|^2 \big)^2 ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation313"><![CDATA[$\lambda_{\Phi_{\bf 32}}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation314"><![CDATA[$w$]]></tex-math></inline-formula> are constants, and
<disp-formula id="ptx175-M2-21"><label>(2.21)</label><tex-math notation="LaTeX" id="Equation21"><![CDATA[
\begin{align}
D_\mu\Phi_{\bf 32}=& \left(\partial_\mu-ig
A_\mu^{SO(11)}\right)\Phi_{\bf 32}=
\left\{\partial_\mu-\frac{ig}{\sqrt{2}}
\sum_{j<k}^{11}A_\mu^{(\,jk)}T_{jk}^{\rm sp}\right\}\Phi_{\bf 32}
,\cr D_v\Phi_{\bf 32}=& \left(\partial_v-ig
A_v^{SO(11)}\right)\Phi_{\bf 32}=
\left\{\partial_v-\frac{ig}{\sqrt{2}}
\sum_{j<k}^{11}A_v^{(\,jk)}T_{jk}^{\rm sp}\right\}\Phi_{\bf 32} .
\end{align}]]></tex-math></disp-formula></p>
<p>We denote the 5D brane scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation315"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> as
<disp-formula id="ptx175-M2-22"><label>(2.22)</label><tex-math notation="LaTeX" id="Equation22"><![CDATA[
\begin{align}
\Phi_{\bf 32}:=
\left(
\begin{array}{c}
\Phi_{\bf 16}\\
\Phi_{\overline{\bf 16}}\\
\end{array}
\right)
\end{align}]]></tex-math></disp-formula>
and assume that the component <inline-formula><tex-math notation="LaTeX" id="ImEquation316"><![CDATA[$({\bf 1,2,\overline{4}})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation317"><![CDATA[$G_{\rm PS}\ (=SU(2)_L\times SU(2)_R\times SU(4)_C)$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation318"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> develops the non-vanishing VEV:
<disp-formula id="ptx175-M2-23"><label>(2.23)</label><tex-math notation="LaTeX" id="Equation23"><![CDATA[
\begin{align}
\langle\Phi_{\bf 16}\rangle=
\left(
\begin{array}{c}
0_4\\
0_4\\
v_4\\
0_4\\
\end{array}
\right)\!,\ \ \
v_4=
\left(
\begin{array}{c}
0\\
0\\
0\\
w\\
\end{array}
\right)\!,\ \
\langle\Phi_{\overline{\bf 16}}\rangle=0,
\end{align}]]></tex-math></disp-formula>
where one component of <inline-formula><tex-math notation="LaTeX" id="ImEquation319"><![CDATA[$({\bf 1,2,\overline{4}})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation320"><![CDATA[$G_{\rm PS}\ (\subset SO(10))$]]></tex-math></inline-formula> corresponds to <inline-formula><tex-math notation="LaTeX" id="ImEquation321"><![CDATA[${\bf 1}_{+5}$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation322"><![CDATA[$SU(5)\times U(1)_Z\ (\subset SO(10))$]]></tex-math></inline-formula>. The VEV breaks <inline-formula><tex-math notation="LaTeX" id="ImEquation323"><![CDATA[$SO(11)$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation324"><![CDATA[$SU(5)$]]></tex-math></inline-formula>. Note that branching rules of <inline-formula><tex-math notation="LaTeX" id="ImEquation325"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation326"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation327"><![CDATA[${\bf 55}$]]></tex-math></inline-formula> representations for <inline-formula><tex-math notation="LaTeX" id="ImEquation328"><![CDATA[$SO(11)\supset SO(10)\supset SU(5)$]]></tex-math></inline-formula> are given by <inline-formula><tex-math notation="LaTeX" id="ImEquation329"><![CDATA[${\bf 32}=({\bf 16})\oplus({\bf \overline{16}})= ({\bf 10}\oplus{\bf \overline{5}}\oplus{\bf 1})\oplus ({\bf \overline{10}}\oplus{\bf 5}\oplus{\bf 1})$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation330"><![CDATA[${\bf 55}=({\bf 45})\oplus({\bf 10})= ({\bf 24}\oplus{\bf 10}\oplus{\bf \overline{10}}\oplus{\bf 1}) \oplus({\bf 5}\oplus{\bf \overline{5}})$]]></tex-math></inline-formula>, respectively. (For further information see, e.g., Refs. [<xref ref-type="bibr" rid="B42">42</xref>&#x2013;<xref ref-type="bibr" rid="B44">44</xref>].)</p>
<p>We also use the 5D brane scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation331"><![CDATA[$\widetilde{\Phi}_{\bf 32}$]]></tex-math></inline-formula> related to <inline-formula><tex-math notation="LaTeX" id="ImEquation332"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> by
<disp-formula id="ptx175-M2-24"><label>(2.24)</label><tex-math notation="LaTeX" id="Equation24"><![CDATA[
\begin{align}
\widetilde{\Phi}_{\bf 32}=
\left(
\begin{array}{c}
-\widehat{R}\Phi_{\overline{\bf 16}}^*\\
\widehat{R}\Phi_{\bf 16}^*\\
\end{array}
\right)\!,\ \ \
\widehat{R}=\sigma^2\otimes\sigma^3\otimes\sigma^2\otimes\sigma^3.
\end{align}]]></tex-math></disp-formula></p>
<p>For <inline-formula><tex-math notation="LaTeX" id="ImEquation333"><![CDATA[$\langle\Phi_{\bf 32}\rangle\not=0$]]></tex-math></inline-formula>,
<disp-formula id="ptx175-M2-25"><label>(2.25)</label><tex-math notation="LaTeX" id="Equation25"><![CDATA[
\begin{align}
\widehat{R}\langle\Phi_{\bf 16}^* \rangle=
\left(
\begin{array}{c}
0_4\\
0_4\\
0_4\\
\tilde{v}_4\\
\end{array}
\right)\!,\ \ \
\tilde{v}_4=
\left(
\begin{array}{c}
0\\
0\\
w^*\\
0\\
\end{array}
\right)\!.
\end{align}]]></tex-math></disp-formula></p>
<p>The combination of the non-vanishing VEV <inline-formula><tex-math notation="LaTeX" id="ImEquation334"><![CDATA[$\langle\Phi_{\bf 32}\rangle$]]></tex-math></inline-formula> on the UV brane (at <inline-formula><tex-math notation="LaTeX" id="ImEquation335"><![CDATA[$y=0$]]></tex-math></inline-formula>) and the orbifold BCs <inline-formula><tex-math notation="LaTeX" id="ImEquation336"><![CDATA[$P_j\ (\,j=0,1,2,3)$]]></tex-math></inline-formula> reduces <inline-formula><tex-math notation="LaTeX" id="ImEquation337"><![CDATA[$SO(11)$]]></tex-math></inline-formula> to the SM gauge group <inline-formula><tex-math notation="LaTeX" id="ImEquation338"><![CDATA[$G_{\rm SM}\ (=SU(3)_C\times SU(2)_L\times U(1)_Y)$]]></tex-math></inline-formula>.</p>
</sec>
<sec id="SEC2.2.3"><title>2.2.3. Action for the singlet brane fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation339"><![CDATA[$\chi_{\bf 1}$]]></tex-math></inline-formula></title>
<p>The action for <inline-formula><tex-math notation="LaTeX" id="ImEquation340"><![CDATA[$\chi_{\bf 1}^\beta (x,v)$]]></tex-math></inline-formula>, which satisfies the symplectic Majorana condition (<xref ref-type="disp-formula" rid="ptx175-M2-11">2.11</xref>), is
<disp-formula id="ptx175-M2-26"><label>(2.26)</label><tex-math notation="LaTeX" id="Equation26"><![CDATA[
\begin{align}
&S_{\rm 5D\ brane}^{\rm brane}=
\int d^6x\sqrt{-\mbox{det}G} \, \delta(y)
{\cal L}_{\chi_{\bf 1}} , \cr
&{\cal L}_{\chi_{\bf 1}}:=
\frac{1}{2}\overline{\chi}_{\bf 1}^\beta
\left(\gamma^\mu\partial_\mu+\gamma^6\partial_v\right)\chi_{\bf 1}^\beta
- \frac{1}{2} M^{\beta \beta'} \overline{\chi}_{\bf 1}^\beta \chi_{\bf 1}^{\beta'} ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation341"><![CDATA[$M^{\beta \beta'}$]]></tex-math></inline-formula> is a constant matrix.</p>
</sec>
<sec id="SEC2.2.4"><title>2.2.4. Brane masses and interactions</title>
<p>On the UV brane there can be <inline-formula><tex-math notation="LaTeX" id="ImEquation342"><![CDATA[$SO(11)$]]></tex-math></inline-formula>-invariant brane interactions among the bulk fermions, the singlet brane fermion, and the brane scalar. We consider
<disp-formula id="ptx175-M2-27"><label>(2.27)</label><tex-math notation="LaTeX" id="Equation27"><![CDATA[
\begin{align}
S_{\rm 5D\ brane}^{\rm fermion}=&
\int d^6x\sqrt{-\mbox{det}G}\delta(y)
\Big({\cal L}_1+{\cal L}_2+{\cal L}_3+{\cal L}_4+{\cal L}_5
+{\cal L}_6+{\cal L}_7 + {\cal L}_8 \Big) , \cr
{\cal L}_1:=&
-\kappa^{\alpha\beta}
\overline{\Psi_{\bf 32}^{\alpha}}\Gamma^a\Phi_{\bf 32}
\cdot(\Psi_{\bf 11}^{\beta})_a
-\kappa^{\alpha\beta*}
(\overline{\Psi_{\bf 11}^{\beta}})_{a}\Phi_{\bf 32}^{\dagger}
\Gamma^a(\Psi_{\bf 32}^{\alpha})_a,\cr
{\cal L}_2:=&
-\widetilde{\kappa}^{\alpha\beta}
\overline{\Psi_{\bf 32}^{\alpha}}\Gamma^a
\widetilde{\Phi}_{\bf 32}\cdot(\Psi_{\bf 11}^{\beta})_a
-\widetilde{\kappa}^{\alpha\beta*}
(\overline{\Psi_{\bf 11}^{\beta}})_{a}
\widetilde{\Phi}_{\bf 32}^{\dagger}\Gamma^a(\Psi_{\bf 32}^{\alpha})_a,\cr
{\cal L}_3:=&
-2\mu_{\bf 11}^{\beta\beta'}
\overline{\Psi_{\bf 11}^{\beta}}\Psi_{\bf 11}^{\beta'},
\cr
{\cal L}_4:=&
-\kappa^{\prime\alpha\beta}
\overline{\Psi_{\bf 32}^{\alpha}}\Gamma^a\Phi_{\bf 32}
\cdot(\Psi_{\bf 11}^{\prime\beta})_a
-\kappa^{\prime\alpha\beta*}
(\overline{\Psi_{\bf 11}^{\prime\beta}})_{a}\Phi_{\bf 32}^{\dagger}
\Gamma^a(\Psi_{\bf 32}^{\alpha})_a,\cr
{\cal L}_5:=&
-\widetilde{\kappa}^{\prime\alpha\beta}
\overline{\Psi_{\bf 32}^{\alpha}}\Gamma^a
\widetilde{\Phi}_{\bf 32}\cdot(\Psi_{\bf 11}^{\prime\beta})_a
-\widetilde{\kappa}^{\prime\alpha\beta*}
(\overline{\Psi_{\bf 11}^{\prime\beta}})_{a}
\widetilde{\Phi}_{\bf 32}^{\dagger}\Gamma^a(\Psi_{\bf 32}^{\alpha})_a,\cr
{\cal L}_6:=&
-2\mu_{\bf 11}^{\prime\beta\beta'}
\overline{\Psi_{\bf 11}^{\prime\beta}}\Psi_{\bf 11}^{\prime\beta'},
\cr
{\cal L}_7:=&
-\widetilde{\kappa}_{\bf 1}^{\alpha \beta}
\overline{\chi_{\bf 1}^\beta}
\widetilde{\Phi}_{\bf 32}^\dagger \Psi_{\bf 32}^{\alpha}
-\widetilde{\kappa}_{\bf 1}^{\alpha \beta *}
\overline{\Psi_{\bf 32}^{\alpha}}
\widetilde{\Phi}_{\bf 32}^{}\chi_{\bf 1}^\beta , \cr
{\cal L}_8:=&
-{\kappa}_{\bf 1}^{\alpha \beta}
\overline{\chi_{\bf 1}^\beta}
{\Phi}_{\bf 32}^\dagger \Psi_{\bf 32}^{\alpha}
-{\kappa}_{\bf 1}^{\alpha \beta *}
\overline{\Psi_{\bf 32}^{\alpha}}
{\Phi}_{\bf 32}^{}\chi_{\bf 1}^\beta ,
\end{align}]]></tex-math></disp-formula>
where the <inline-formula><tex-math notation="LaTeX" id="ImEquation343"><![CDATA[$\kappa$]]></tex-math></inline-formula>s and <inline-formula><tex-math notation="LaTeX" id="ImEquation344"><![CDATA[$\mu$]]></tex-math></inline-formula>s are coupling constants. Note that <inline-formula><tex-math notation="LaTeX" id="ImEquation345"><![CDATA[$\overline{\Psi_{\bf 11}^{\alpha}}\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation346"><![CDATA[$\overline{\Psi_{\bf 11}^{\prime\beta}}\Psi_{\bf 11}^{\alpha}$]]></tex-math></inline-formula> are forbidden by the parity assignment of <inline-formula><tex-math notation="LaTeX" id="ImEquation347"><![CDATA[$\overline{\Psi_{\bf 11}^{\alpha}}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation348"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula> shown in <xref ref-type="table" rid="T3">Table 3</xref>.</p>
</sec>
<sec id="SEC2.2.5"><title>2.2.5. Mass terms for fermions on the UV brane</title>
<p>From the action in Eqs. (<xref ref-type="disp-formula" rid="ptx175-M2-26">2.26</xref>) and (<xref ref-type="disp-formula" rid="ptx175-M2-27">2.27</xref>), the quadratic mass terms on the UV brane for the bulk and brane fermions with <inline-formula><tex-math notation="LaTeX" id="ImEquation349"><![CDATA[$\langle\Phi_{\rm 32}\rangle\not=0$]]></tex-math></inline-formula> are given by
<disp-formula id="ptx175-M2-28"><label>(2.28)</label><tex-math notation="LaTeX" id="Equation28"><![CDATA[
\begin{align}
S_{\rm brane\ mass}^{\rm fermion}=&
\int d^4x\int_0^{2\pi R_6}dv
\left(
\sum_{j=1}^8 {\cal L}_j^m
+{\cal L}_{\chi_{\bf 1}}^m\right)\!,
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M2-29"><label>(2.29)</label><tex-math notation="LaTeX" id="Equation29"><![CDATA[
\begin{align}
{\cal L}_1^m=&
-\mu_1^{\alpha\beta}
\left\{-i2
(\overline{\check{\hat{e}}^{\prime\alpha}}\check{\hat{E}}^{\beta}
+\overline{\check{\hat{\nu}}^{\prime\alpha}}\check{\hat{N}}^{\beta})
- 2\overline{\check{d}^{\prime\alpha}}\check{D}^{\beta}
+\sqrt{2}\overline{\check{\hat{\nu}}^{\alpha}}\check{S}^{\beta}\right\}
+\mbox{h.c.},
\cr
{\cal L}_2^m=&
-\widetilde{\mu}_1^{\alpha\beta}
\left\{i2
(\overline{\check{e}^{\alpha}}\check{E}^{\beta}
+\overline{\check{\nu}^{\alpha}}\check{N}^{\beta})
-2\overline{\check{\hat{d}}^{\alpha}}\check{\hat{D}}^{\beta}
+\sqrt{2}\overline{\check{\nu}^{\prime\alpha}}\check{S}^{\beta}\right\}
+\mbox{h.c},
\cr
{\cal L}_3^m=&
-2 \mu_{\bf 11}^{\beta\beta'}
\bigg\{
\overline{\check{D}^{\beta}}\check{D}^{\beta'}
+\overline{\check{\hat{D}}^{\beta}}\check{\hat{D}}^{\beta'}
+\overline{\check{E}^{\beta}}\check{E}^{\beta'}
+\overline{\check{\hat{E}}^{\beta}}\check{\hat{E}}^{\beta'} \cr
&\hspace{6cm}
+\overline{\check{N}^{\beta}}\check{N}^{\beta'}
+\overline{\check{\hat{N}}^{\beta}}\check{\hat{N}}^{\beta'}
+\overline{\check{S}^{\beta}}\check{S}^{\beta'}
\bigg\},
\cr
{\cal L}_4^m=&
-\mu_2^{\alpha\beta}
\left\{-i
2(\overline{\check{\hat{e}}^{\prime\alpha}}\check{\hat{E}}^{\prime\beta}
+\overline{\check{\hat{\nu}}^{\prime\alpha}}\check{\hat{N}}^{\prime\beta})
-2\overline{\check{d}^{\prime\alpha}}\check{D}^{\prime\beta}
+\sqrt{2}\overline{\check{\hat{\nu}}^{\alpha}}\check{S}^{\prime\beta}
\right\}
+\mbox{h.c.},
\cr
{\cal L}_5^m=&
-\widetilde{\mu}_2^{\alpha\beta}
\left\{i2
(\overline{\check{e}^{\alpha}}\check{E}^{\prime\beta}
+\overline{\check{\nu}^{\alpha}}\check{N}^{\prime\beta})
-2\overline{\check{\hat{d}}^{\alpha}}\check{\hat{D}}^{\prime\beta}
+\sqrt{2}\overline{\check{\nu}^{\prime\alpha}}\check{S}^{\prime\beta}
\right\}
+\mbox{h.c},
\cr
{\cal L}_6^m=&
-2 \mu_{\bf 11}^{\prime\beta\beta'}
\bigg\{
\overline{\check{D}^{\prime\beta}}\check{D}^{\prime\beta'}
+\overline{\check{\hat{D}}^{\prime\beta}}\check{\hat{D}}^{\prime\beta'}
+\overline{\check{E}^{\prime\beta}}\check{E}^{\prime\beta'}
+\overline{\check{\hat{E}}^{\prime\beta}}\check{\hat{E}}^{\prime\beta'} \cr
&\hspace{6cm}
+\overline{\check{N}^{\prime\beta}}\check{N}^{\prime\beta'}
+\overline{\check{\hat{N}}^{\prime\beta}}\check{\hat{N}}^{\prime\beta'}
+\overline{\check{S}^{\prime\beta}}\check{S}^{\prime\beta'}
\bigg\},
\cr
{\cal L}_7^m=&
- \frac{m_B^{\alpha \beta}}{\sqrt{k}} \,
( \overline{\chi^\beta} \, \check{\nu}^{\prime\alpha} +
\overline{\check{\nu}^{\prime\alpha}} \, \chi^\beta ) , \cr
{\cal L}_8^m=&
- \frac{\hat m_B^{\alpha \beta} }{\sqrt{k}} \,
( \overline{\chi^\beta} \, \check{\hat \nu}^{\alpha} +
\overline{\check{\hat \nu}^{\alpha}} \, \chi^\beta ) , \cr
{\cal L}_{\chi_{\bf 1}}^m=&
-\frac{1}{2} M^{\beta \beta'} \, \overline{\chi^\beta} \, \chi^{\beta '} .
\end{align}]]></tex-math></disp-formula></p>
<p>Here, <inline-formula><tex-math notation="LaTeX" id="ImEquation350"><![CDATA[$2\mu_{1}^{\alpha\beta}:=\sqrt{2}\kappa^{\alpha\beta}w$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation351"><![CDATA[$2\widetilde{\mu}_{1}^{\alpha\beta}:= \sqrt{2}\widetilde{\kappa}^{\alpha\beta}w$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation352"><![CDATA[$2\mu_{2}^{\alpha\beta}:=\sqrt{2}\kappa^{\prime\alpha\beta}w$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation353"><![CDATA[$2\widetilde{\mu}_{2}^{\alpha\beta}:= \sqrt{2}\widetilde{\kappa}^{\prime\alpha\beta}w$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation354"><![CDATA[$m_B^{\alpha \beta}/\sqrt{k} :=\widetilde{\kappa}_{\bf 1}^{\alpha \beta}w$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation355"><![CDATA[$\hat m_B^{\alpha \beta }/ \sqrt{k} := {\kappa}_{\bf 1}^{\alpha \beta}w$]]></tex-math></inline-formula>. For the sake of simplicity <inline-formula><tex-math notation="LaTeX" id="ImEquation356"><![CDATA[$\chi_{\bf 1}^\beta$]]></tex-math></inline-formula> has been denoted as <inline-formula><tex-math notation="LaTeX" id="ImEquation357"><![CDATA[$\chi^\beta$]]></tex-math></inline-formula>. All the <inline-formula><tex-math notation="LaTeX" id="ImEquation358"><![CDATA[$\mu$]]></tex-math></inline-formula> parameters are dimensionless quantities, whereas <inline-formula><tex-math notation="LaTeX" id="ImEquation359"><![CDATA[$m_B^{\alpha \beta}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation360"><![CDATA[$\hat m_B^{\alpha \beta}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation361"><![CDATA[$M^{\beta \beta'}$]]></tex-math></inline-formula> have the dimension of mass.</p>
</sec>
<sec id="SEC2.2.6"><title>2.2.6. Mass terms for gauge bosons on the UV brane</title>
<p>By replacing <inline-formula><tex-math notation="LaTeX" id="ImEquation362"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> with its VEV <inline-formula><tex-math notation="LaTeX" id="ImEquation363"><![CDATA[$\langle\Phi_{\bf 32}\rangle$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-20">2.20</xref>), the mass terms for the 4D components of the <inline-formula><tex-math notation="LaTeX" id="ImEquation364"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge fields <inline-formula><tex-math notation="LaTeX" id="ImEquation365"><![CDATA[$A_\mu$]]></tex-math></inline-formula> can be read off as
<disp-formula id="ptx175-M2-30"><label>(2.30)</label><tex-math notation="LaTeX" id="Equation30"><![CDATA[
\begin{align}
{\cal L}_{\rm brane\ mass}^{\rm 4D\ gauge}=&
-\frac{g^2w^2}{8}
\bigg\{
\left(A_\mu^{15}-A_\mu^{26}\right)^2
+\left(A_\mu^{16}+A_\mu^{25}\right)^2
\nonumber\\
&
+\left(A_\mu^{17}-A_\mu^{28}\right)^2
+\left(A_\mu^{18}+A_\mu^{27}\right)^2
+\left(A_\mu^{19}-A_\mu^{2,10}\right)^2
+\left(A_\mu^{1,10}+A_\mu^{29}\right)^2
\nonumber\\
&
+\left(A_\mu^{35}+A_\mu^{46}\right)^2
+\left(A_\mu^{36}-A_\mu^{45}\right)^2
+\left(A_\mu^{37}+A_\mu^{48}\right)^2
+\left(A_\mu^{38}-A_\mu^{47}\right)^2
\nonumber\\
&
+\left(A_\mu^{39}+A_\mu^{4,10}\right)^2
+\left(A_\mu^{3,10}-A_\mu^{49}\right)^2
+\left(A_\mu^{57}-A_\mu^{68}\right)^2
+\left(A_\mu^{58}+A_\mu^{67}\right)^2
\nonumber\\
&
+\left(A_\mu^{59}-A_\mu^{6,10}\right)^2
+\left(A_\mu^{5,10}+A_\mu^{69}\right)^2
+\left(A_\mu^{79}-A_\mu^{8,10}\right)^2
+\left(A_\mu^{7,10}+A_\mu^{89}\right)^2
\nonumber\\
&
+\left(A_\mu^{23}-A_\mu^{14}\right)^2
+\left(A_\mu^{31}-A_\mu^{24}\right)^2
+\left(A_\mu^{12}-A_\mu^{34}+A_\mu^{56}+A_\mu^{78}+A_\mu^{9,10}\right)^2
\nonumber\\
&
+\left(A_\mu^{1,11}\right)^2
+\left(A_\mu^{2,11}\right)^2
+\left(A_\mu^{3,11}\right)^2
+\left(A_\mu^{4,11}\right)^2
+\left(A_\mu^{5,11}\right)^2
\nonumber\\
&
+\left(A_\mu^{6,11}\right)^2
+\left(A_\mu^{7,11}\right)^2
+\left(A_\mu^{8,11}\right)^2
+\left(A_\mu^{9,11}\right)^2
+\left(A_\mu^{10,11}\right)^2
\bigg\},
\end{align}]]></tex-math></disp-formula>
where the notation is the same as in Ref. [<xref ref-type="bibr" rid="B35">35</xref>]. We omit them here. All the 31 components of <inline-formula><tex-math notation="LaTeX" id="ImEquation366"><![CDATA[$SO(11)/SU(5)$]]></tex-math></inline-formula> obtain large brane masses via <inline-formula><tex-math notation="LaTeX" id="ImEquation367"><![CDATA[$\langle\Phi_{\bf 32}\rangle\not=0$]]></tex-math></inline-formula>. Each mass term changes the fifth-dimensional BCs on the UV brane for the corresponding fields. Similarly, from the action in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-20">2.20</xref>), we find the mass terms for the sixth-dimensional component of the <inline-formula><tex-math notation="LaTeX" id="ImEquation368"><![CDATA[$A_v$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M2-31"><label>(2.31)</label><tex-math notation="LaTeX" id="Equation31"><![CDATA[
\begin{align}
{\cal L}_{\rm brane\ mass}^{\rm 6th\ dim\ gauge}=&
-\frac{g^2w^2}{8}
\bigg\{
\left(A_v^{15}-A_v^{26}\right)^2
+\left(A_v^{16}+A_v^{25}\right)^2
\nonumber\\
&
+\left(A_v^{17}-A_v^{28}\right)^2
+\left(A_v^{18}+A_v^{27}\right)^2
+\left(A_v^{19}-A_v^{2,10}\right)^2
+\left(A_v^{1,10}+A_v^{29}\right)^2
\nonumber\\
&
+\left(A_v^{35}+A_v^{46}\right)^2
+\left(A_v^{36}-A_v^{45}\right)^2
+\left(A_v^{37}+A_v^{48}\right)^2
+\left(A_v^{38}-A_v^{47}\right)^2
\nonumber\\
&
+\left(A_v^{39}+A_v^{4,10}\right)^2
+\left(A_v^{3,10}-A_v^{49}\right)^2
+\left(A_v^{57}-A_v^{68}\right)^2
+\left(A_v^{58}+A_v^{67}\right)^2
\nonumber\\
&
+\left(A_v^{59}-A_v^{6,10}\right)^2
+\left(A_v^{5,10}+A_v^{69}\right)^2
+\left(A_v^{79}-A_v^{8,10}\right)^2
+\left(A_v^{7,10}+A_v^{89}\right)^2
\nonumber\\
&
+\left(A_v^{23}-A_v^{14}\right)^2
+\left(A_v^{31}-A_v^{24}\right)^2
+\left(A_v^{12}-A_v^{34}+A_v^{56}+A_v^{78}+A_v^{9,10}\right)^2
\nonumber\\
&
+\left(A_v^{1,11}\right)^2
+\left(A_v^{2,11}\right)^2
+\left(A_v^{3,11}\right)^2
+\left(A_v^{4,11}\right)^2
+\left(A_v^{5,11}\right)^2
\nonumber\\
&
+\left(A_v^{6,11}\right)^2
+\left(A_v^{7,11}\right)^2
+\left(A_v^{8,11}\right)^2
+\left(A_v^{9,11}\right)^2
+\left(A_v^{10,11}\right)^2
\bigg\}.
\end{align}]]></tex-math></disp-formula></p>
<p>These mass terms give masses for the corresponding fields by changing the BCs. We shall see more details in <xref ref-type="sec" rid="SEC3">Section 3</xref>.</p>
</sec>
</sec>
<sec id="SEC2.3"><title>2.3. EW Higgs boson and twisted gauge</title>
<p>The orbifold BCs break <inline-formula><tex-math notation="LaTeX" id="ImEquation369"><![CDATA[$SO(11)$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation370"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula>, and, further, the non-vanishing VEV of the 5D brane scalar <inline-formula><tex-math notation="LaTeX" id="ImEquation371"><![CDATA[$\Phi_{\rm 32}$]]></tex-math></inline-formula> reduces <inline-formula><tex-math notation="LaTeX" id="ImEquation372"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation373"><![CDATA[$G_{\rm SM}$]]></tex-math></inline-formula>. The bilinear terms of the action of gauge fields in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-13">2.13</xref>) are written down as
<disp-formula id="ptx175-M2-32"><label>(2.32)</label><tex-math notation="LaTeX" id="Equation32"><![CDATA[
\begin{align}
&\left(S_{\rm bulk}^{\rm gauge}\right)_{\rm quadratic} \cr
&\quad{}= \int d^4xdv \frac{dz}{kz^2} \,
\sum_{j<k} \Bigg[ \frac{1}{2}A_{\lambda}^{(\,jk)} \bigg\{ \hat
\eta^{\lambda \rho} \big(\Box+k^2{\cal P}_5+\partial_v^2 \big)-
\Big( 1-\frac{1}{\xi} \Big) \partial^\lambda \partial^\rho \bigg\}
A_\rho^{(\,jk)} \cr &\qquad{}
+\frac{1}{2}k^2A_z^{(\,jk)} \left(\Box+\xi k^2{\cal
P}_z+\partial_v^2\right)A_z^{(\,jk)} +\overline{c}^{(\,jk)}
\left(\Box+\xi k^2{\cal P}_5+\partial_v^2\right)c^{(\,jk)}
\Bigg], \cr & \hat \eta_{\lambda
\rho} = \hat \eta^{\lambda \rho} = {\rm diag} (-1, +1, +1, +1, +1) , \cr & \Box =\eta^{\mu\nu}\partial_\mu\partial_\nu,\
\partial^\lambda = \hat \eta^{\lambda \rho} \partial_\rho,
{\cal P}_5 =z^2\frac{\partial}{\partial z}
\frac{1}{z^2}\frac{\partial}{\partial z},\
{\cal P}_z =\frac{\partial}{\partial z}
z^2\frac{\partial}{\partial z}\frac{1}{z^2}.
\end{align}]]></tex-math></disp-formula></p>
<p>Here, <inline-formula><tex-math notation="LaTeX" id="ImEquation374"><![CDATA[$\lambda, \rho$]]></tex-math></inline-formula> run over <inline-formula><tex-math notation="LaTeX" id="ImEquation375"><![CDATA[$0,1,2,3,6$]]></tex-math></inline-formula> so that <inline-formula><tex-math notation="LaTeX" id="ImEquation376"><![CDATA[$A_\lambda \hat \eta^{\lambda \rho} B_\rho = A_\mu \eta^{\mu \nu} B_\nu + A_v B_v$]]></tex-math></inline-formula>. The fourth- and sixth-dimensional components <inline-formula><tex-math notation="LaTeX" id="ImEquation377"><![CDATA[$A_\mu$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation378"><![CDATA[$A_v$]]></tex-math></inline-formula> have additional bilinear terms that come from the 5D brane scalar term <inline-formula><tex-math notation="LaTeX" id="ImEquation379"><![CDATA[${\cal L}_{\Phi_{\bf 32}}$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-20">2.20</xref>) with <inline-formula><tex-math notation="LaTeX" id="ImEquation380"><![CDATA[$\langle\Phi_{\bf 32}\rangle\not=0$]]></tex-math></inline-formula>. The parity assignments of <inline-formula><tex-math notation="LaTeX" id="ImEquation381"><![CDATA[$A_\mu$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation382"><![CDATA[$A_z$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation383"><![CDATA[$A_v$]]></tex-math></inline-formula> are summarized in <xref ref-type="table" rid="T1">Table 1</xref>. The components <inline-formula><tex-math notation="LaTeX" id="ImEquation384"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation385"><![CDATA[$A_z$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation386"><![CDATA[$A_v$]]></tex-math></inline-formula> have parity <inline-formula><tex-math notation="LaTeX" id="ImEquation387"><![CDATA[$(+,+)$]]></tex-math></inline-formula>. The components <inline-formula><tex-math notation="LaTeX" id="ImEquation388"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation389"><![CDATA[$A_v$]]></tex-math></inline-formula> obtain masses through the brane mass terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-31">2.31</xref>), so only the components <inline-formula><tex-math notation="LaTeX" id="ImEquation390"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation391"><![CDATA[$A_z$]]></tex-math></inline-formula>, which is identified with the SM Higgs scalar fields, have zero modes. Three of them are absorbed by <inline-formula><tex-math notation="LaTeX" id="ImEquation392"><![CDATA[$W$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation393"><![CDATA[$Z$]]></tex-math></inline-formula> bosons. The remaining neutral component, the observed Higgs boson, also becomes massive by radiative corrections through the Hosotani mechanism as shown below. We note that the components <inline-formula><tex-math notation="LaTeX" id="ImEquation394"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation395"><![CDATA[$A_v$]]></tex-math></inline-formula> get additional masses of order <inline-formula><tex-math notation="LaTeX" id="ImEquation396"><![CDATA[$g R_6^{-1}$]]></tex-math></inline-formula> by radiative corrections at one loop. The components <inline-formula><tex-math notation="LaTeX" id="ImEquation397"><![CDATA[$({\bf 3,1,1})$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation398"><![CDATA[$({\bf 1,3,1})$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation399"><![CDATA[$({\bf 1,1,15})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation400"><![CDATA[$A_\mu$]]></tex-math></inline-formula> have parity <inline-formula><tex-math notation="LaTeX" id="ImEquation401"><![CDATA[$(+,+)$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation402"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> is spontaneously broken to <inline-formula><tex-math notation="LaTeX" id="ImEquation403"><![CDATA[$G_{\rm SM}$]]></tex-math></inline-formula> by the non-vanishing VEV of the component <inline-formula><tex-math notation="LaTeX" id="ImEquation404"><![CDATA[$({\bf 1,2,\overline{4}})$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation405"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula>.</p>
<p>The zero modes of <inline-formula><tex-math notation="LaTeX" id="ImEquation406"><![CDATA[$A_z$]]></tex-math></inline-formula> are physical degrees of freedom which cannot be gauged away. By using the KK mode expansion discussed in <xref ref-type="sec" rid="SECA.1">Appendix A.1</xref>, the sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation407"><![CDATA[$n=0$]]></tex-math></inline-formula> modes (<inline-formula><tex-math notation="LaTeX" id="ImEquation408"><![CDATA[$v$]]></tex-math></inline-formula>-independent modes) of <inline-formula><tex-math notation="LaTeX" id="ImEquation409"><![CDATA[$A_z^{a\, 11}$]]></tex-math></inline-formula> are expanded, in terms of mode functions <inline-formula><tex-math notation="LaTeX" id="ImEquation410"><![CDATA[$\big\{h_n^{(++)}(z)\big\}$]]></tex-math></inline-formula> for the parity <inline-formula><tex-math notation="LaTeX" id="ImEquation411"><![CDATA[$(+,+)$]]></tex-math></inline-formula> boundary condition, as
<disp-formula id="ptx175-M2-33"><label>(2.33)</label><tex-math notation="LaTeX" id="Equation33"><![CDATA[
\begin{align}
A_z^{a \, 11}=&
\frac{1}{\sqrt{2\pi R_6}} \bigg\{
\phi_H^{a (0)}(x)u_H(z)+
\sum_{n=1}^{\infty}\phi_H^{a(n)}(x)h_n^{(++)}(z) \bigg\} \ \
(a=1,2,3,4),
\end{align}]]></tex-math></disp-formula>
where the zero mode function in the fifth dimension is given by <inline-formula><tex-math notation="LaTeX" id="ImEquation412"><![CDATA[$u_H (z) = [3/k (z_L^3 -1) ]^{1/2} \, z^2$]]></tex-math></inline-formula>. The four-component real field <inline-formula><tex-math notation="LaTeX" id="ImEquation413"><![CDATA[$\phi_H^{a (0)}(x)$]]></tex-math></inline-formula> is identified with the EW Higgs doublet field in the SM. We will show later that the <inline-formula><tex-math notation="LaTeX" id="ImEquation414"><![CDATA[$\phi_H:=\phi_H^{ 4 (0)}$]]></tex-math></inline-formula> dynamically obtain the non-vanishing VEV <inline-formula><tex-math notation="LaTeX" id="ImEquation415"><![CDATA[$\langle\phi_H\rangle\not=0$]]></tex-math></inline-formula>. The VEV breaks <inline-formula><tex-math notation="LaTeX" id="ImEquation416"><![CDATA[$G_{\rm SM}$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation417"><![CDATA[$SU(3)_C\times U(1)_{\rm EM}$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation418"><![CDATA[$\phi_H^{1,2,3 (0)}$]]></tex-math></inline-formula> are absorbed by <inline-formula><tex-math notation="LaTeX" id="ImEquation419"><![CDATA[$W$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation420"><![CDATA[$Z$]]></tex-math></inline-formula> bosons.</p>
<p>Under a general gauge transformation <inline-formula><tex-math notation="LaTeX" id="ImEquation421"><![CDATA[$A_M'=\Omega A_M \Omega^{-1}+(i/g)\Omega\partial_M\Omega^{-1}$]]></tex-math></inline-formula>, new gauge potentials satisfy the following new BCs:
<disp-formula id="ptx175-M2-34"><label>(2.34)</label><tex-math notation="LaTeX" id="Equation34"><![CDATA[
\begin{align}
\begin{pmatrix} A_\mu'\\ A_y'\\ A_v' \end{pmatrix} (x,y_j-y,v_j-v)& =
P_j'
\begin{pmatrix} A_\mu'\\ -A_y'\\ -A_v' \end{pmatrix}(x,y_j+y,v_j+v) P_j^{\prime -1}\cr
&\quad{} +\frac{i}{g} P_j'
\begin{pmatrix} \partial_\mu\\ -\partial_y\\ -\partial_v \end{pmatrix} P_j^{\prime -1}, \cr
&\hskip .5cm
P_j' =\Omega(x,y_j-y,v_j-v)P_j\Omega(x,y_j+y,v_j+v)^{-1} .
\end{align}]]></tex-math></disp-formula></p>
<p>It is convenient to work in the twisted gauge discussed, e.g., in Ref. [<xref ref-type="bibr" rid="B35">35</xref>]. With the following large gauge transformation <inline-formula><tex-math notation="LaTeX" id="ImEquation422"><![CDATA[$\Omega(y;\alpha)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation423"><![CDATA[$\hat{\theta}_H$]]></tex-math></inline-formula> is transformed to <inline-formula><tex-math notation="LaTeX" id="ImEquation424"><![CDATA[$\hat{\theta}_H'=0$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M2-35"><label>(2.35)</label><tex-math notation="LaTeX" id="Equation35"><![CDATA[
\begin{align}
\Omega(y)& = \mbox{exp}
\left\{i\frac{g\theta_H f_H}{\sqrt{4\pi R_{6}}}\int_y^{L}dy
\tilde{u}_H(y)T_{4,11}\right\}
=\mbox{exp}
\left\{i\theta(z)T_{4,11}\right\}\!, \cr
\theta(z) & = \theta_H\frac{z_L^3-z^3}{z_L^3-1}\ \
\mbox{for}\ 1\leq z\leq z_L.
\end{align}]]></tex-math></disp-formula></p>
<p>The new BC matrices <inline-formula><tex-math notation="LaTeX" id="ImEquation425"><![CDATA[$P_j'$]]></tex-math></inline-formula> are given by
<disp-formula id="ptx175-M2-36"><label>(2.36)</label><tex-math notation="LaTeX" id="Equation36"><![CDATA[
\begin{align}
\tilde{P}_j& = \Omega(0)^{2}P_j=e^{2i\theta_H T_{4,11}}P_j\ \
(\,j=0,2),\cr \tilde{P}_k& = P_k\ \ (k=1,3).
\end{align}]]></tex-math></disp-formula></p>
<p>For fields in the <inline-formula><tex-math notation="LaTeX" id="ImEquation426"><![CDATA[$SO(11)$]]></tex-math></inline-formula> vector and spinor representations, the BC matrices are given by
<disp-formula id="ptx175-M2-37"><label>(2.37)</label><tex-math notation="LaTeX" id="Equation37"><![CDATA[
\begin{align}
\tilde{P}_0^{\rm vec}& =
\begin{cases}
\begin{pmatrix} \cos\theta_H&-\sin\theta_H\cr -\sin\theta_H&-\cos\theta_H \end{pmatrix}
&\mbox{in the 4-11 subspace},\cr
I_3&\mbox{in the 1,2,3 subspace}, \cr
- I_6&\mbox{in the
5-10 subspace},
\end{cases} \cr
\tilde{P}_2^{\rm vec}& =
\begin{cases}
\begin{pmatrix} \cos\theta_H&-\sin\theta_H\cr -\sin\theta_H&-\cos\theta_H \end{pmatrix}
&\mbox{in the 4-11 subspace},\cr
I_9&\mbox{otherwise},
\end{cases} \cr
\tilde{P}_0^{\rm sp}& =
\begin{pmatrix}
\pm \cos\frac{\theta_H}{2}& - i \sin\frac{\theta_H}{2}\\
i\sin\frac{\theta_H}{2}& \mp\cos\frac{\theta_H}{2} \end{pmatrix}
\quad
\mbox{for}\
\bigg\{ \,
\begin{matrix} \psi \cr \hat \psi \end{matrix}\!, \cr
\tilde{P}_2^{\rm sp}& =
\begin{pmatrix}
\cos\frac{\theta_H}{2}&\mp i \sin\frac{\theta_H}{2}\\
\pm i\sin\frac{\theta_H}{2}&-\cos\frac{\theta_H}{2} \end{pmatrix}
\quad
\mbox{for}\
\bigg\{ \,
\begin{matrix} \psi \cr \hat \psi \end{matrix}\!,
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M2-38"><label>(2.38)</label><tex-math notation="LaTeX" id="Equation38"><![CDATA[
\begin{align}
{\psi}& =
\left(
\begin{array}{c}
{\nu}\\
{\nu}'\\
\end{array}
\right)\!,\
\left(
\begin{array}{c}
{e}\\
{e}'\\
\end{array}
\right)\!,\
\left(
\begin{array}{c}
{u}_j\\
{u}_j'\\
\end{array}
\right)\!,\
\left(
\begin{array}{c}
{d}_j\\
{d}_j'\\
\end{array}
\right)\!,\nonumber\\
\hat{\psi}& =
\left(
\begin{array}{c}
\hat{\nu}\\
\hat{\nu}'\\
\end{array}
\right)\!,\
\left(
\begin{array}{c}
\hat{e}\\
\hat{e}'\\
\end{array}
\right)\!,\
\left(
\begin{array}{c}
\hat{u}_j\\
\hat{u}_j'\\
\end{array}
\right)\!,\
\left(
\begin{array}{c}
\hat{d}_j\\
\hat{d}_j'\\
\end{array}
\right)\!.
\end{align}]]></tex-math></disp-formula></p>
</sec>
</sec>
<sec id="SEC3"><title>3. Mass spectrum of bosons</title>
<p>In this section we examine the spectrum of gauge fields, particularly for the sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation427"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes (<inline-formula><tex-math notation="LaTeX" id="ImEquation428"><![CDATA[$v$]]></tex-math></inline-formula>-independent modes), because we are interested in the mass spectrum of the SM particles and the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation429"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula>. In the following we omit the modes which are odd under the sixth-dimensional loop translation <inline-formula><tex-math notation="LaTeX" id="ImEquation430"><![CDATA[$U_6 = P_2 P_0 = P_3 P_1 = -1$]]></tex-math></inline-formula>. Those modes have masses <inline-formula><tex-math notation="LaTeX" id="ImEquation431"><![CDATA[$\ge \frac{1}{2} m_{{\rm KK}_6}$]]></tex-math></inline-formula>.</p>
<p>The BCs for gauge fields in the absence of the brane terms are given by
<disp-formula id="ptx175-M3-1"><label>(3.1)</label><tex-math notation="LaTeX" id="Equation39"><![CDATA[
\begin{align}
&\left\{
\begin{array}{ll@{\hspace*{26pt}}l}
N:\ &\frac{\partial}{\partial z}A_\mu=0\ &\mbox{for parity}=+,\cr
D:\ &A_\mu=0\ &\mbox{for parity}=-,
\end{array}
\right.\cr
&\left\{
\begin{array}{lll}
N:\ &\frac{\partial}{\partial z}\left(\frac{1}{z^2}A_z\right)=0\ &
\mbox{for parity}=+, \cr
D:\ &A_z=0\ &\mbox{for parity}=-,
\end{array}
\right. \cr
&\left\{
\begin{array}{lll}
N:\ &\frac{\partial}{\partial z}A_v=0\ &\mbox{for parity}=+, \cr
D:\ &A_v=0\ &\mbox{for parity}=-
\end{array}
\right.
\end{align}]]></tex-math></disp-formula>
at <inline-formula><tex-math notation="LaTeX" id="ImEquation432"><![CDATA[$z=1 \ (y=0)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation433"><![CDATA[$z=z_L \ (y=L)$]]></tex-math></inline-formula>. In the presence of the brane mass terms <inline-formula><tex-math notation="LaTeX" id="ImEquation434"><![CDATA[$| g A_\mu \langle\Phi_{\rm 32} \rangle |^2$]]></tex-math></inline-formula> on the UV brane, the Neumann BC <inline-formula><tex-math notation="LaTeX" id="ImEquation435"><![CDATA[$N$]]></tex-math></inline-formula> is modified to an effective Dirichlet BC <inline-formula><tex-math notation="LaTeX" id="ImEquation436"><![CDATA[$D_{\rm eff}$]]></tex-math></inline-formula> for the fifth-dimensional zero mode of the <inline-formula><tex-math notation="LaTeX" id="ImEquation437"><![CDATA[$SO(11)/SU(5)$]]></tex-math></inline-formula> components of <inline-formula><tex-math notation="LaTeX" id="ImEquation438"><![CDATA[$A_\mu$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation439"><![CDATA[$A_v$]]></tex-math></inline-formula> [<xref ref-type="bibr" rid="B35">35</xref>]. The equation of motion for <inline-formula><tex-math notation="LaTeX" id="ImEquation440"><![CDATA[$A_\mu^a$]]></tex-math></inline-formula> in the <inline-formula><tex-math notation="LaTeX" id="ImEquation441"><![CDATA[$y$]]></tex-math></inline-formula>-coordinate is given in the form
<disp-formula id="ptx175-M3-2"><label>(3.2)</label><tex-math notation="LaTeX" id="Equation40"><![CDATA[
\begin{align}
\bigg( \Box + e^{\sigma (y)}\frac{\partial}{\partial y} e^{-3\sigma (y)} \frac{\partial}{\partial y}
+ \frac{\partial^2}{\partial v^2} - \frac{g^2 w^2}{2} \, \delta (y) \bigg) A_\mu^a
-
\bigg(1 - \frac{1}{\xi} \bigg) \partial_\mu ( \partial^\nu A_\nu^a + \partial_v A_v^a) = \cdots ,
\end{align}]]></tex-math></disp-formula>
where the right-hand side involves interaction terms. Suppose that <inline-formula><tex-math notation="LaTeX" id="ImEquation442"><![CDATA[$A_\mu^a$]]></tex-math></inline-formula> is parity even, namely <inline-formula><tex-math notation="LaTeX" id="ImEquation443"><![CDATA[$A_\mu^a (x, -y, -v) = + A_\mu^a (x, y, v)$]]></tex-math></inline-formula>. Note that <inline-formula><tex-math notation="LaTeX" id="ImEquation444"><![CDATA[$A_v^a$]]></tex-math></inline-formula> is parity odd so that <inline-formula><tex-math notation="LaTeX" id="ImEquation445"><![CDATA[$\partial_v A_v^a$]]></tex-math></inline-formula> is parity even. By integrating the equation <inline-formula><tex-math notation="LaTeX" id="ImEquation446"><![CDATA[$\int_{-\epsilon}^\epsilon dy \cdots$]]></tex-math></inline-formula> and taking the limit <inline-formula><tex-math notation="LaTeX" id="ImEquation447"><![CDATA[$\epsilon \rightarrow 0$]]></tex-math></inline-formula>, one finds <inline-formula><tex-math notation="LaTeX" id="ImEquation448"><![CDATA[$\partial A_\mu^a / \partial y |_{y=\epsilon} = (g^2 w^2/4) A_\mu^a |_{y=0}$]]></tex-math></inline-formula>. In the <inline-formula><tex-math notation="LaTeX" id="ImEquation449"><![CDATA[$z$]]></tex-math></inline-formula>-coordinate the Neumann condition is changed to the effective Dirichlet condition
<disp-formula id="ptx175-M3-3"><label>(3.3)</label><tex-math notation="LaTeX" id="Equation41"><![CDATA[
\begin{align}
D_{\rm eff} (\omega) : \left( \frac{\partial}{\partial z} - \omega \right) \, A_\mu^a =0 ,
\omega = \frac{g^2 w^2}{4k} \quad {\rm at} z=1^+ .
\end{align}]]></tex-math></disp-formula></p>
<p>Note that in the current six-dimensional model the mass dimensions of <inline-formula><tex-math notation="LaTeX" id="ImEquation450"><![CDATA[$g$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation451"><![CDATA[$w$]]></tex-math></inline-formula> are <inline-formula><tex-math notation="LaTeX" id="ImEquation452"><![CDATA[$[g] = M^{-1}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation453"><![CDATA[$[w] = M^{3/2}$]]></tex-math></inline-formula>, so that <inline-formula><tex-math notation="LaTeX" id="ImEquation454"><![CDATA[$\omega$]]></tex-math></inline-formula> is dimensionless. When <inline-formula><tex-math notation="LaTeX" id="ImEquation455"><![CDATA[$\omega \not=0$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation456"><![CDATA[$A_\mu^a$]]></tex-math></inline-formula> develops a cusp at <inline-formula><tex-math notation="LaTeX" id="ImEquation457"><![CDATA[$y=0$]]></tex-math></inline-formula>. The lowest mode of <inline-formula><tex-math notation="LaTeX" id="ImEquation458"><![CDATA[$A_\mu^a$]]></tex-math></inline-formula> becomes massive, with mass <inline-formula><tex-math notation="LaTeX" id="ImEquation459"><![CDATA[$O(m_{\rm KK_5})$]]></tex-math></inline-formula>. The value of <inline-formula><tex-math notation="LaTeX" id="ImEquation460"><![CDATA[$\omega$]]></tex-math></inline-formula> depends on the brane mass terms. The BCs for the gauge fields are summarized in <xref ref-type="table" rid="T5">Table 5</xref>.</p>
<p><table-wrap id="T5" orientation="portrait" position="float"><label>Table 5</label><caption><p>Venn diagram for the symmetry-breaking pattern and summary of the BCs <inline-formula><tex-math notation="LaTeX" id="ImEquation461"><![CDATA[$\left( {\matrix{ {{P_2}} \hfill & {{P_3}} \hfill \cr {{P_0}} \hfill & {{P_1}} \hfill \cr } } \right)$]]></tex-math></inline-formula> of the <inline-formula><tex-math notation="LaTeX" id="ImEquation462"><![CDATA[${SO}(11)$]]></tex-math></inline-formula> gauge field. <inline-formula><tex-math notation="LaTeX" id="ImEquation463"><![CDATA[$N$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation464"><![CDATA[$D$]]></tex-math></inline-formula> stand for Neumann and Dirichlet conditions, respectively. <inline-formula><tex-math notation="LaTeX" id="ImEquation465"><![CDATA[$D_{\rm eff}$]]></tex-math></inline-formula> represents the effective Dirichlet condition given by Eq. (<xref ref-type="disp-formula" rid="ptx175-M3-3">3.3</xref>).</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" orientation="portrait" position="float" xlink:href="ptx175TF5.tif"/>
</table-wrap></p>
<p>In the twisted gauge <inline-formula><tex-math notation="LaTeX" id="ImEquation466"><![CDATA[$\tilde{A}_M=\Omega(z)A_M\Omega(z)^{-1} +\frac{i}{g}\Omega(z)\partial_M\Omega(z)^{-1}$]]></tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX" id="ImEquation467"><![CDATA[$\Omega (z) = e^{i \theta (z) T_{4, 11}}$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation468"><![CDATA[$\theta (z)$]]></tex-math></inline-formula> is defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-35">2.35</xref>). The <inline-formula><tex-math notation="LaTeX" id="ImEquation469"><![CDATA[$(k,4)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation470"><![CDATA[$(k,11)$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation471"><![CDATA[$(4,11)$]]></tex-math></inline-formula> components of the <inline-formula><tex-math notation="LaTeX" id="ImEquation472"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge fields are changed as
<disp-formula id="ptx175-M3-4"><label>(3.4)</label><tex-math notation="LaTeX" id="Equation42"><![CDATA[
\begin{align}
A_M^{k4}& =
\cos\theta(z)\tilde{A}_M^{k4}-\sin\theta(z)\tilde{A}_{M}^{k,11}\
(k\not=4,11),\cr
A_M^{k,11}& =
\sin\theta(z)\tilde{A}_M^{k4}+\sin\theta(z)\tilde{A}_{M}^{k,11},\cr
A_z^{4,11}& =
\tilde{A}_z^{4,11}-\frac{\sqrt{2}}{g}\theta^{\prime}(z)
=\tilde{A}_z^{4,11}+\frac{3\sqrt{2}}{g} \frac{z^2}{z_L^3-1} \,\theta_H,
\end{align}]]></tex-math></disp-formula>
while the other components remain unchanged. As in Ref. [<xref ref-type="bibr" rid="B35">35</xref>], the BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation473"><![CDATA[$z=z_L$]]></tex-math></inline-formula> determine wave functions for <inline-formula><tex-math notation="LaTeX" id="ImEquation474"><![CDATA[$\tilde{A}_\mu$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation475"><![CDATA[$\tilde{A}_z$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation476"><![CDATA[$\tilde{A}_v$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M3-5"><label>(3.5)</label><tex-math notation="LaTeX" id="Equation43"><![CDATA[
\begin{align}
&\tilde{A}_\mu\ \ N:\ C(z;\lambda),\ D:\ S(z;\lambda),\cr
&\tilde{A}_z\ \ N:\ S'(z;\lambda),\ D:\ C'(z;\lambda),\cr
&\tilde{A}_v\ \ N:\ S'(z;\lambda),\ D:\ C'(z;\lambda),
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation477"><![CDATA[$C(z;\lambda)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation478"><![CDATA[$S(z;\lambda)$]]></tex-math></inline-formula> are defined by Eqs. (<xref ref-type="disp-formula" rid="ptx175-MA-2">A.2</xref>) and (<xref ref-type="disp-formula" rid="ptx175-MA-3">A.3</xref>) in <xref ref-type="sec" rid="SECA.1">Appendix A.1</xref>.</p>
<p>From the BCs for the gauge fields summarized in <xref ref-type="table" rid="T5">Table 5</xref>, the components (2) <inline-formula><tex-math notation="LaTeX" id="ImEquation479"><![CDATA[$SU(5)/G_{\rm SM}$]]></tex-math></inline-formula>, (4) <inline-formula><tex-math notation="LaTeX" id="ImEquation480"><![CDATA[$SO(10)/(SU(5)\cup G_{\rm PS})$]]></tex-math></inline-formula>, and (6) <inline-formula><tex-math notation="LaTeX" id="ImEquation481"><![CDATA[$SO(7)/SO(6)$]]></tex-math></inline-formula> are parity odd under the sixth-dimensional loop translation <inline-formula><tex-math notation="LaTeX" id="ImEquation482"><![CDATA[$U_6$]]></tex-math></inline-formula>, while the components (1) <inline-formula><tex-math notation="LaTeX" id="ImEquation483"><![CDATA[$G_{\rm SM}$]]></tex-math></inline-formula>, (3) <inline-formula><tex-math notation="LaTeX" id="ImEquation484"><![CDATA[$G_{\rm PS}/G_{\rm SM}$]]></tex-math></inline-formula>, and (5) <inline-formula><tex-math notation="LaTeX" id="ImEquation485"><![CDATA[$SO(5)/SO(4)$]]></tex-math></inline-formula> are parity even under <inline-formula><tex-math notation="LaTeX" id="ImEquation486"><![CDATA[$U_6$]]></tex-math></inline-formula>. Thus, we find that the low-lying modes of the components (2), (4), and (6) of <inline-formula><tex-math notation="LaTeX" id="ImEquation487"><![CDATA[$A_M$]]></tex-math></inline-formula> have masses of <inline-formula><tex-math notation="LaTeX" id="ImEquation488"><![CDATA[$O(m_{\rm KK_6})$]]></tex-math></inline-formula>, while the low-lying modes of the components (1), (3), and (5) of <inline-formula><tex-math notation="LaTeX" id="ImEquation489"><![CDATA[$A_M$]]></tex-math></inline-formula> have masses less than or around <inline-formula><tex-math notation="LaTeX" id="ImEquation490"><![CDATA[$O(m_{\rm KK_5})$]]></tex-math></inline-formula>.</p>
<p>We summarize the formulas determining the mass spectrum of the sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation491"><![CDATA[$n=0$]]></tex-math></inline-formula> modes of <inline-formula><tex-math notation="LaTeX" id="ImEquation492"><![CDATA[$A_M$]]></tex-math></inline-formula>, for which the arguments in Ref. [<xref ref-type="bibr" rid="B35">35</xref>] remain intact. We use the same notation as Ref. [<xref ref-type="bibr" rid="B35">35</xref>]. <inline-formula><tex-math notation="LaTeX" id="ImEquation493"><![CDATA[$A_M^{a_L}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation494"><![CDATA[$A_M^{a_R}$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation495"><![CDATA[$a_L, a_R=1,2,3$]]></tex-math></inline-formula>) stand for <inline-formula><tex-math notation="LaTeX" id="ImEquation496"><![CDATA[$SU(2)_L$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation497"><![CDATA[$SU(2)_R$]]></tex-math></inline-formula> components of <inline-formula><tex-math notation="LaTeX" id="ImEquation498"><![CDATA[$A_M$]]></tex-math></inline-formula>, respectively.</p>
<p><list list-type="bullet">
<list-item><p><inline-formula><tex-math notation="LaTeX" id="ImEquation499"><![CDATA[$A_\mu$]]></tex-math></inline-formula> components.
<list list-type="number">
<list-item><p>(i) <inline-formula><tex-math notation="LaTeX" id="ImEquation500"><![CDATA[$(\tilde{A}_\mu^{a_L},\tilde{A}_\mu^{a_R},\tilde{A}_\mu^{a,11})$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation501"><![CDATA[$(a=1,2)$]]></tex-math></inline-formula>: For <inline-formula><tex-math notation="LaTeX" id="ImEquation502"><![CDATA[$W$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation503"><![CDATA[$W_R$]]></tex-math></inline-formula> towers, there are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation504"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, and there are also fifth-dimensional zero modes in the absence of brane terms. In the presence of the brane terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-30">2.30</xref>), their mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-6"><label>(3.6)</label><tex-math notation="LaTeX" id="Equation44"><![CDATA[
\begin{align}
2C'(SC'+\lambda\sin^2\theta_H)-\omega C(2SC'+\lambda\sin^2\theta_H)=0,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation505"><![CDATA[$\omega:=g^2w^2/4k$]]></tex-math></inline-formula>. Let us denote the average magnitude of <inline-formula><tex-math notation="LaTeX" id="ImEquation506"><![CDATA[$C(1;\lambda)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation507"><![CDATA[$C'(1;\lambda)$]]></tex-math></inline-formula> by <inline-formula><tex-math notation="LaTeX" id="ImEquation508"><![CDATA[$\overline{C}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation509"><![CDATA[$\overline{C'}$]]></tex-math></inline-formula>, respectively. When <inline-formula><tex-math notation="LaTeX" id="ImEquation510"><![CDATA[$\omega \overline{C} \gg \overline{C'}$]]></tex-math></inline-formula>, the spectrum is determined by
<disp-formula id="ptx175-M3-7"><label>(3.7)</label><tex-math notation="LaTeX" id="Equation45"><![CDATA[
\begin{align}
W\,\hbox{tower:}\ &2S(1;\lambda)C'(1;\lambda)+\lambda\sin^2\theta_H=0 , \cr
W_R\,\hbox{tower:}\ &C(1;\lambda)=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>Indeed, in the typical cases examined below, <inline-formula><tex-math notation="LaTeX" id="ImEquation511"><![CDATA[$\overline{C}/\overline{C'} \sim 10^{6}$]]></tex-math></inline-formula> so that the condition <inline-formula><tex-math notation="LaTeX" id="ImEquation512"><![CDATA[$\omega \overline{C} \gg \overline{C'}$]]></tex-math></inline-formula> is well satisfied for <inline-formula><tex-math notation="LaTeX" id="ImEquation513"><![CDATA[$w \gg (m_{{\rm KK}_5})^{3/2}$]]></tex-math></inline-formula>. The mass of the <inline-formula><tex-math notation="LaTeX" id="ImEquation514"><![CDATA[$W$]]></tex-math></inline-formula> boson <inline-formula><tex-math notation="LaTeX" id="ImEquation515"><![CDATA[$m_W = m_{W^{(0)}}$]]></tex-math></inline-formula> is given by
<disp-formula id="ptx175-M3-8"><label>(3.8)</label><tex-math notation="LaTeX" id="Equation46"><![CDATA[
\begin{align}
m_W\simeq\sqrt{\frac{3}{2}}kz_L^{-3/2}\sin\theta_H
=\frac{\sqrt{3}\sin\theta_H}{\sqrt{2}\pi\sqrt{z_L}}\, m_{\rm KK_5}.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(ii) <inline-formula><tex-math notation="LaTeX" id="ImEquation516"><![CDATA[$(\tilde{A}_\mu^{3_L},\tilde{C}_\mu ,\tilde{B}_\mu^Y,\tilde{A}_\mu^{3,11})$]]></tex-math></inline-formula>: For <inline-formula><tex-math notation="LaTeX" id="ImEquation517"><![CDATA[$\gamma$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation518"><![CDATA[$Z$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation519"><![CDATA[$Z_R$]]></tex-math></inline-formula> towers, there are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation520"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, and there are also fifth-dimensional zero modes in the absence of brane terms. Here, <inline-formula><tex-math notation="LaTeX" id="ImEquation521"><![CDATA[$C_\mu=\sqrt{2/5}A_\mu^{3_R}+\sqrt{3/5} A_\mu^{0_C}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation522"><![CDATA[$B_\mu^Y =\sqrt{3/5}A_\mu^{3_R}-\sqrt{2/5} A_\mu^{0_C}$]]></tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX" id="ImEquation523"><![CDATA[$A_\mu^{0_C}=(A_\mu^{56}+A_\mu^{78}+A_\mu^{9\ 10})/\sqrt{3}$]]></tex-math></inline-formula>. In the presence of the brane terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-30">2.30</xref>), the mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-9"><label>(3.9)</label><tex-math notation="LaTeX" id="Equation47"><![CDATA[
\begin{align}
C'\left\{2C'(SC'+\lambda\sin^2\theta_H)
-\omega C(5SC'+4\lambda\sin^2\theta_H)\right\}=0.
\end{align}]]></tex-math></disp-formula></p>
<p>For <inline-formula><tex-math notation="LaTeX" id="ImEquation524"><![CDATA[$\omega \overline{C} \gg \overline{C'}$]]></tex-math></inline-formula>,
<disp-formula id="ptx175-M3-10"><label>(3.10)</label><tex-math notation="LaTeX" id="Equation48"><![CDATA[
\begin{align}
\gamma\,\hbox{tower:}\ &C'(1;\lambda)=0,\cr
Z~\hbox{tower:}\ &5S(1;\lambda)C'(1;\lambda)+4\lambda\sin^2\theta_H=0,\cr
Z_R ~\hbox{tower:}\ &C(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p>
<p>The mass of the <inline-formula><tex-math notation="LaTeX" id="ImEquation525"><![CDATA[$Z$]]></tex-math></inline-formula> boson <inline-formula><tex-math notation="LaTeX" id="ImEquation526"><![CDATA[$m_Z=m_{Z^{(0)}}$]]></tex-math></inline-formula> is given by
<disp-formula id="ptx175-M3-11"><label>(3.11)</label><tex-math notation="LaTeX" id="Equation49"><![CDATA[
\begin{align}
m_Z\simeq\sqrt{\frac{12}{5}}kz_L^{-3/2}\sin\theta_H=
\frac{m_W}{\cos \theta_W},\ \ \ \sin^2 \theta_W = \frac{3}{8} .
\end{align}]]></tex-math></disp-formula></p>
<p>The relation for the <inline-formula><tex-math notation="LaTeX" id="ImEquation527"><![CDATA[$Z$]]></tex-math></inline-formula> tower in Eq. (<xref ref-type="disp-formula" rid="ptx175-M3-10">3.10</xref>) can be written, by using <inline-formula><tex-math notation="LaTeX" id="ImEquation528"><![CDATA[$\cos^2 \theta_W = \frac{5}{8}$]]></tex-math></inline-formula>, as
<disp-formula id="ptx175-M3-12"><label>(3.12)</label><tex-math notation="LaTeX" id="Equation50"><![CDATA[
\begin{align}
Z~\hbox{tower:} ~ &2 S(1;\lambda)C'(1;\lambda)
+\frac{\lambda}{\cos^2 \theta_W} \sin^2\theta_H=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>The value <inline-formula><tex-math notation="LaTeX" id="ImEquation529"><![CDATA[$\sin^2 \theta_W = \frac{3}{8}$]]></tex-math></inline-formula> is valid at the GUT scale. The gauge coupling constants evolve as the energy scale, following the renormalization group equation (RGE). It is not clear whether <inline-formula><tex-math notation="LaTeX" id="ImEquation530"><![CDATA[$\sin^2 \theta_W$]]></tex-math></inline-formula> evolves to the observed value at low energies as there are KK modes which do not respect <inline-formula><tex-math notation="LaTeX" id="ImEquation531"><![CDATA[$SU(5)$]]></tex-math></inline-formula> below the GUT scale. We leave solving the RGE for future investigation. When we evaluate the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation532"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula> at the EW scale in <xref ref-type="sec" rid="SEC5">Sect. 5</xref>, we adopt the formula in Eq. (<xref ref-type="disp-formula" rid="ptx175-M3-12">3.12</xref>), where the observed value, <inline-formula><tex-math notation="LaTeX" id="ImEquation533"><![CDATA[$\sin^2 \theta_W \simeq 0.2312$]]></tex-math></inline-formula>, at the EW scale is inserted.</p></list-item>
<list-item><p>(iii) <inline-formula><tex-math notation="LaTeX" id="ImEquation534"><![CDATA[$\tilde A_\mu^{4, 11}$]]></tex-math></inline-formula>: For the <inline-formula><tex-math notation="LaTeX" id="ImEquation535"><![CDATA[$\hat A^{4}$]]></tex-math></inline-formula> tower, there are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation536"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, but there are no fifth-dimensional zero modes. The mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-13"><label>(3.13)</label><tex-math notation="LaTeX" id="Equation51"><![CDATA[
\begin{align}
\hat{A}^{4}~\hbox{tower:}\ &S(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(iv) For <inline-formula><tex-math notation="LaTeX" id="ImEquation537"><![CDATA[$SU(3)_C$]]></tex-math></inline-formula> gluons, there are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation538"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, and there are also fifth-dimensional zero modes which correspond to 4D gluons. The mass spectrum of the fifth-dimensional massive modes is given by
<disp-formula id="ptx175-M3-14"><label>(3.14)</label><tex-math notation="LaTeX" id="Equation52"><![CDATA[
\begin{align}
\hbox{gluon tower:}\ &C'(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(v) For <inline-formula><tex-math notation="LaTeX" id="ImEquation539"><![CDATA[$X$]]></tex-math></inline-formula>-gluons, there are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation540"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, and there are also fifth-dimensional zero modes in the absence of brane terms. In the presence of the brane terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-30">2.30</xref>), the mass spectrum of the fifth-dimensional modes is given by
<disp-formula id="ptx175-M3-15"><label>(3.15)</label><tex-math notation="LaTeX" id="Equation53"><![CDATA[
\begin{align}
X\hbox{-gluon tower:}\ &C'(1;\lambda)-\omega C(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p>
<p>For <inline-formula><tex-math notation="LaTeX" id="ImEquation541"><![CDATA[$\omega \overline{C} \gg \overline{C'}$]]></tex-math></inline-formula>,
<disp-formula id="ptx175-M3-16"><label>(3.16)</label><tex-math notation="LaTeX" id="Equation54"><![CDATA[
\begin{align}
X\hbox{-gluon tower:}\ &C(1;\lambda)\simeq 0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(vi) For <inline-formula><tex-math notation="LaTeX" id="ImEquation542"><![CDATA[$X$]]></tex-math></inline-formula>-bosons, there are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation543"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
<list-item><p>(vii) For <inline-formula><tex-math notation="LaTeX" id="ImEquation544"><![CDATA[$X'$]]></tex-math></inline-formula>-bosons, there are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation545"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
<list-item><p>(viii) For <inline-formula><tex-math notation="LaTeX" id="ImEquation546"><![CDATA[$Y, Y'$]]></tex-math></inline-formula>-bosons, there are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation547"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
</list></p></list-item>
<list-item><p><inline-formula><tex-math notation="LaTeX" id="ImEquation548"><![CDATA[$A_z$]]></tex-math></inline-formula> components.
<list list-type="number">
<list-item><p>(i) <inline-formula><tex-math notation="LaTeX" id="ImEquation549"><![CDATA[$A_z^{ab}\ (1\leq a< b\leq 3)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation550"><![CDATA[$A_z^{jk}\ (5\leq j<k\leq 10)$]]></tex-math></inline-formula>: There are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation551"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, but there are no fifth-dimensional zero modes. The mass spectrum of the fifth-dimensional modes is given by
<disp-formula id="ptx175-M3-17"><label>(3.17)</label><tex-math notation="LaTeX" id="Equation55"><![CDATA[
\begin{align}
C'(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(ii) <inline-formula><tex-math notation="LaTeX" id="ImEquation552"><![CDATA[$A_z^{a4},\ A_z^{a\, 11}\ (a=1,2,3)$]]></tex-math></inline-formula>: There are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation553"><![CDATA[$n=0$]]></tex-math></inline-formula> modes for <inline-formula><tex-math notation="LaTeX" id="ImEquation554"><![CDATA[$A_z^{a4},\ A_z^{a\, 11}\ (a=1,2,3)$]]></tex-math></inline-formula>. There are no fifth-dimensional zero modes for <inline-formula><tex-math notation="LaTeX" id="ImEquation555"><![CDATA[$A_z^{a4}$]]></tex-math></inline-formula>, while there are fifth-dimensional zero modes for <inline-formula><tex-math notation="LaTeX" id="ImEquation556"><![CDATA[$A_z^{a\, 11}$]]></tex-math></inline-formula>. The mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-18"><label>(3.18)</label><tex-math notation="LaTeX" id="Equation56"><![CDATA[
\begin{align}
S(1;\lambda)C'(1;\lambda)+\lambda\sin^2\theta_H=0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(iii) <inline-formula><tex-math notation="LaTeX" id="ImEquation557"><![CDATA[$A_z^{4,11}$]]></tex-math></inline-formula>: Higgs tower. There are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation558"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, and there is also a fifth-dimensional zero mode. The mass spectrum of the fifth-dimensional modes is given by
<disp-formula id="ptx175-M3-19"><label>(3.19)</label><tex-math notation="LaTeX" id="Equation57"><![CDATA[
\begin{align}
\hbox{Higgs tower:}\ &S(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(iv) <inline-formula><tex-math notation="LaTeX" id="ImEquation559"><![CDATA[$A_z^{ak}\ (a=1,2,3,\ k=5,\ldots,10)$]]></tex-math></inline-formula>: There are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation560"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
<list-item><p>(v) <inline-formula><tex-math notation="LaTeX" id="ImEquation561"><![CDATA[$A_z^{k4}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation562"><![CDATA[$A_z^{k11}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation563"><![CDATA[$(k= 5,\ldots,10)$]]></tex-math></inline-formula>: There are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation564"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
</list></p></list-item>
<list-item><p><inline-formula><tex-math notation="LaTeX" id="ImEquation565"><![CDATA[$A_v$]]></tex-math></inline-formula> components.
<list list-type="number">
<list-item><p>(i) <inline-formula><tex-math notation="LaTeX" id="ImEquation566"><![CDATA[$A_v^{ab}\ (1\leq a< b\leq 3)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation567"><![CDATA[$A_v^{jk}\ (5\leq j<k\leq 10)$]]></tex-math></inline-formula>: There are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation568"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, but there are no fifth-dimensional zero modes. The mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-20"><label>(3.20)</label><tex-math notation="LaTeX" id="Equation58"><![CDATA[
\begin{align}
C'(1;\lambda)=0.
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(ii) <inline-formula><tex-math notation="LaTeX" id="ImEquation569"><![CDATA[$A_v^{a4},\ A_v^{a\ 11}\ (a=1,2,3)$]]></tex-math></inline-formula>: There are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation570"><![CDATA[$n=0$]]></tex-math></inline-formula> modes for <inline-formula><tex-math notation="LaTeX" id="ImEquation571"><![CDATA[$A_v^{a4}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation572"><![CDATA[$A_v^{a\ 11}\ (a=1,2,3)$]]></tex-math></inline-formula>. There are no fifth-dimensional zero modes for <inline-formula><tex-math notation="LaTeX" id="ImEquation573"><![CDATA[$A_v^{a4}$]]></tex-math></inline-formula>, while there are fifth-dimensional zero modes for <inline-formula><tex-math notation="LaTeX" id="ImEquation574"><![CDATA[$A_v^{a\ 11}\ (a=1,2,3)$]]></tex-math></inline-formula> in the absence of brane terms. In the presence of the brane terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-31">2.31</xref>), one finds that at <inline-formula><tex-math notation="LaTeX" id="ImEquation575"><![CDATA[$z=1$]]></tex-math></inline-formula>,
<disp-formula id="ptx175-M3-21"><label>(3.21)</label><tex-math notation="LaTeX" id="Equation59"><![CDATA[
\begin{align}
\left(
\begin{array}{cc}
\cos\theta_H C'&-\sin\theta_H S'\\
\sin\theta_H (C-\omega C')&\cos\theta_H (S-\omega S')\\
\end{array}
\right)
\left(
\begin{array}{c}
\beta_{a4}^{v}\\
\beta_{a11}^{v}\\
\end{array}
\right)=0.
\end{align}]]></tex-math></disp-formula></p>
<p>Thus, the mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-22"><label>(3.22)</label><tex-math notation="LaTeX" id="Equation60"><![CDATA[
\begin{align}
C'(1;\lambda)\left\{S(1;\lambda)-\omega S'(1;\lambda)\right\}
+\lambda\sin^2\theta_H=0.
\end{align}]]></tex-math></disp-formula></p>
<p>The average magnitude of <inline-formula><tex-math notation="LaTeX" id="ImEquation576"><![CDATA[$S(1;\lambda)$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation577"><![CDATA[$=\overline{S}$]]></tex-math></inline-formula>) is much larger than that of <inline-formula><tex-math notation="LaTeX" id="ImEquation578"><![CDATA[$S'(1;\lambda)$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation579"><![CDATA[$=\overline{S'}$]]></tex-math></inline-formula>). In typical cases <inline-formula><tex-math notation="LaTeX" id="ImEquation580"><![CDATA[$\overline{S}/\overline{S'} \sim 10^{4}$]]></tex-math></inline-formula>. Hence the spectrum is determined by
<disp-formula id="ptx175-M3-23"><label>(3.23)</label><tex-math notation="LaTeX" id="Equation61"><![CDATA[
\begin{align}
{C'(1;\lambda)S(1;\lambda) + \lambda\sin^2\theta_H=0. }
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(iii) <inline-formula><tex-math notation="LaTeX" id="ImEquation581"><![CDATA[$A_v^{4,11}$]]></tex-math></inline-formula>: <inline-formula><tex-math notation="LaTeX" id="ImEquation582"><![CDATA[$A_v$]]></tex-math></inline-formula>-Higgs tower. There are sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation583"><![CDATA[$n=0$]]></tex-math></inline-formula> modes, and there are also fifth-dimensional zero modes in the absence of brane terms. In the presence of the brane terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-31">2.31</xref>), the mass spectra of the fifth-dimensional modes are given by
<disp-formula id="ptx175-M3-24"><label>(3.24)</label><tex-math notation="LaTeX" id="Equation62"><![CDATA[
\begin{align}
A_v\hbox{-Higgs tower:}\ &
S(1;\lambda)-\omega S'(1;\lambda)=0,
\end{align}]]></tex-math></disp-formula>
which is well approximated by
<disp-formula id="ptx175-M3-25"><label>(3.25)</label><tex-math notation="LaTeX" id="Equation63"><![CDATA[
\begin{align}
A_v\hbox{-Higgs tower:}\ &
{S(1;\lambda)=0.}
\end{align}]]></tex-math></disp-formula></p>
<p>The components <inline-formula><tex-math notation="LaTeX" id="ImEquation584"><![CDATA[$A_v^{a \, 11}$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation585"><![CDATA[$a=1,\ldots, 4$]]></tex-math></inline-formula>) acquire large one-loop corrections of order <inline-formula><tex-math notation="LaTeX" id="ImEquation586"><![CDATA[$g_w M_{{\rm KK}_6}$]]></tex-math></inline-formula> to their masses so that their contributions to the <inline-formula><tex-math notation="LaTeX" id="ImEquation587"><![CDATA[$\theta_H$]]></tex-math></inline-formula>-dependent part of <inline-formula><tex-math notation="LaTeX" id="ImEquation588"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula> become negligible and may be dropped.</p></list-item>
<list-item><p>(iv) <inline-formula><tex-math notation="LaTeX" id="ImEquation589"><![CDATA[$A_v^{ak}\ (a=1,2,3,\ k=5,\ldots,10)$]]></tex-math></inline-formula>: There are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation590"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
<list-item><p>(v) <inline-formula><tex-math notation="LaTeX" id="ImEquation591"><![CDATA[$A_v^{k4}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation592"><![CDATA[$A_v^{k11}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation593"><![CDATA[$(k= 5,\ldots,10)$]]></tex-math></inline-formula>: There are no sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation594"><![CDATA[$n=0$]]></tex-math></inline-formula> modes.</p></list-item>
</list></p></list-item>
</list></p>
</sec>
<sec id="SEC4"><title>4. Mass spectrum of fermions</title>
<p>In this section we determine the mass spectrum of quarks and leptons. For up-type quarks, there are no brane interactions on the UV brane <inline-formula><tex-math notation="LaTeX" id="ImEquation595"><![CDATA[$(y=0)$]]></tex-math></inline-formula> as in the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation596"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT. For down-type quarks, charged leptons, and neutrinos, both 6D bulk mass terms and brane mass terms on the UV brane are important to reproduce the observed mass spectra.</p>
<p>In the following, we will calculate mass spectra for one set of a 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation597"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation598"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> Weyl fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation599"><![CDATA[$\Psi_{\bf 32}^{\alpha}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation600"><![CDATA[$(\alpha=1,2, ~{\rm or}~3)$]]></tex-math></inline-formula>, 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation601"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation602"><![CDATA[${\bf 11}$]]></tex-math></inline-formula> Dirac fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation603"><![CDATA[$\Psi_{\bf 11}^{\beta}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation604"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation605"><![CDATA[$(\beta=\alpha)$]]></tex-math></inline-formula>, and 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation606"><![CDATA[$SO(11)$]]></tex-math></inline-formula> singlet <inline-formula><tex-math notation="LaTeX" id="ImEquation607"><![CDATA[$\chi_{\bf 1}^\alpha$]]></tex-math></inline-formula>, which is identified with one generation of the SM quarks and leptons. We will also discuss mass spectra for the 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation608"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation609"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> Weyl fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation610"><![CDATA[$\Psi_{\bf 32}^{\alpha=4}$]]></tex-math></inline-formula> denoted by <inline-formula><tex-math notation="LaTeX" id="ImEquation611"><![CDATA[$\Psi_{\bf 32}^{\prime}$]]></tex-math></inline-formula>. We will omit the superscripts <inline-formula><tex-math notation="LaTeX" id="ImEquation612"><![CDATA[$\alpha$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation613"><![CDATA[$\beta$]]></tex-math></inline-formula>. We are interested in EW-scale physics and we keep only the sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation614"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes because the masses of sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation615"><![CDATA[$n\not=0$]]></tex-math></inline-formula> modes are much larger than <inline-formula><tex-math notation="LaTeX" id="ImEquation616"><![CDATA[$O(m_{\rm KK_5})$]]></tex-math></inline-formula> as <inline-formula><tex-math notation="LaTeX" id="ImEquation617"><![CDATA[$m_{\rm KK_6}\gg m_{\rm KK_5}$]]></tex-math></inline-formula>. That is, we will discuss mass spectra for the <inline-formula><tex-math notation="LaTeX" id="ImEquation618"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation619"><![CDATA[$({\bf 2,1,4})$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation620"><![CDATA[$({\bf 1,2,4})$]]></tex-math></inline-formula> components of <inline-formula><tex-math notation="LaTeX" id="ImEquation621"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula>, the <inline-formula><tex-math notation="LaTeX" id="ImEquation622"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation623"><![CDATA[$({\bf 1,1,6})$]]></tex-math></inline-formula> components of <inline-formula><tex-math notation="LaTeX" id="ImEquation624"><![CDATA[$\Psi_{\bf 11}$]]></tex-math></inline-formula>, the <inline-formula><tex-math notation="LaTeX" id="ImEquation625"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation626"><![CDATA[$({\bf 2,2,1})$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation627"><![CDATA[$({\bf 1,1,1})$]]></tex-math></inline-formula> components of <inline-formula><tex-math notation="LaTeX" id="ImEquation628"><![CDATA[$\Psi_{\bf 11}^{\prime}$]]></tex-math></inline-formula>, and the <inline-formula><tex-math notation="LaTeX" id="ImEquation629"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation630"><![CDATA[$({\bf 2,1,4})$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation631"><![CDATA[$({\bf 1,2,4})$]]></tex-math></inline-formula> components of <inline-formula><tex-math notation="LaTeX" id="ImEquation632"><![CDATA[$\Psi_{\bf 32}^{\prime}$]]></tex-math></inline-formula>. We will denote <inline-formula><tex-math notation="LaTeX" id="ImEquation633"><![CDATA[$c_{\Psi_{\bf 32}}^{\alpha\not=4}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation634"><![CDATA[$c_{\Psi_{\bf 11}}^{\beta}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation635"><![CDATA[$c_{\Psi_{\bf 11}}^{\prime\beta}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation636"><![CDATA[$c_{\Psi_{\bf 32}}^{\alpha=4}$]]></tex-math></inline-formula> as <inline-formula><tex-math notation="LaTeX" id="ImEquation637"><![CDATA[$c_0$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation638"><![CDATA[$c_1$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation639"><![CDATA[$c_2$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation640"><![CDATA[$c_0^{\prime}$]]></tex-math></inline-formula>, respectively. In this paper we assume that <inline-formula><tex-math notation="LaTeX" id="ImEquation641"><![CDATA[$c_0, c_1, c_0^{\prime} \ge 0$]]></tex-math></inline-formula> while <inline-formula><tex-math notation="LaTeX" id="ImEquation642"><![CDATA[$c_2$]]></tex-math></inline-formula> can be negative. The calculation method employed in the following is the same as employed in the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation643"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT discussed in Ref. [<xref ref-type="bibr" rid="B35">35</xref>].</p>
<sec id="SEC4.1"><title>4.1. Up-type quark</title>
<p>(i) <inline-formula><tex-math notation="LaTeX" id="ImEquation644"><![CDATA[$Q_{\rm EM}=+2/3$]]></tex-math></inline-formula>: <inline-formula><tex-math notation="LaTeX" id="ImEquation645"><![CDATA[$u_j,u_j'$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation646"><![CDATA[$(\Psi_{\bf 32})$]]></tex-math></inline-formula></p>
<p>From the action in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-19">2.19</xref>), we find the equations of motion for up-type quarks with <inline-formula><tex-math notation="LaTeX" id="ImEquation647"><![CDATA[$\gamma_{6D}^7 = +1$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M4-1"><label>(4.1)</label><tex-math notation="LaTeX" id="Equation64"><![CDATA[
\begin{align}
-i\delta
\left(
\begin{array}{c}
u_{+L}^{\dagger}\\
u_{+L}^{\prime\dagger}\\
\end{array}
\right):\ &
\left(-k\hat{D}_-(c_{0})+i\partial_v\right)
\left(
\begin{array}{c}
\check{u}_{+R}\\
\check{u}_{+R}^{\prime}\\
\end{array}
\right)
+\sigma^\mu\partial_\mu
\left(
\begin{array}{c}
\check{u}_{+L}\\
\check{u}_{+L}^{\prime}\\
\end{array}
\right)
=0,
 \cr
i\delta
\left(
\begin{array}{c}
u_{+R}^{\dagger}\\
u_{+R}^{\prime\dagger}\\
\end{array}
\right):\ &
\overline{\sigma}^\mu\partial_\mu
\left(
\begin{array}{c}
\check{u}_{+R}\\
\check{u}_{+R}^{\prime}\\
\end{array}
\right)
+\left(-k\hat{D}_+(c_{0})+i\partial_v\right)
\left(
\begin{array}{c}
\check{u}_{+L}\\
\check{u}_{+L}^{\prime}\\
\end{array}
\right)
=0.
\end{align}]]></tex-math></disp-formula></p>
<p>Here, <inline-formula><tex-math notation="LaTeX" id="ImEquation648"><![CDATA[$\sigma^\mu = (I_2, \vec \sigma)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation649"><![CDATA[$\overline{\sigma}^\mu = (-I_2, \vec \sigma)$]]></tex-math></inline-formula>, and
<disp-formula id="ptx175-M4-2"><label>(4.2)</label><tex-math notation="LaTeX" id="Equation65"><![CDATA[
\begin{align}
\hat{D}_\pm (c) &= \pm \left( \frac{\partial}{\partial z} + i \theta ' (z) T_{4,11} \right) + \frac{c}{z} , \cr
T_{4,11} &= \begin{cases} ~ \frac{1}{2} \tau_1 &{\rm for~} \psi, \cr
- \frac{1}{2} \tau_1 &{\rm for~} \hat \psi, \end{cases}
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation650"><![CDATA[$\psi$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation651"><![CDATA[$\hat \psi$]]></tex-math></inline-formula> are the pairs defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-38">2.38</xref>). For an up-type quark with <inline-formula><tex-math notation="LaTeX" id="ImEquation652"><![CDATA[$\gamma_{6D}^7 = -1$]]></tex-math></inline-formula>, namely for the top quark, the equations become
<disp-formula id="ptx175-M4-3"><label>(4.3)</label><tex-math notation="LaTeX" id="Equation66"><![CDATA[
\begin{align}
&
\left( k\hat{D}_+ (c_{0}) - i\partial_v\right)
\left(
\begin{array}{c}
\check{u}_{-R}\\
\check{u}_{-R}^{\prime}\\
\end{array}
\right)
+\sigma^\mu\partial_\mu
\left(
\begin{array}{c}
\check{u}_{-L}\\
\check{u}_{-L}^{\prime}\\
\end{array}
\right)
=0,
 \cr
&
\overline{\sigma}^\mu\partial_\mu
\left(
\begin{array}{c}
\check{u}_{-R}\\
\check{u}_{-R}^{\prime}\\
\end{array}
\right)
+\left( k\hat{D}_-(c_{0}) - i\partial_v\right)
\left(
\begin{array}{c}
\check{u}_{-L}\\
\check{u}_{-L}^{\prime}\\
\end{array}
\right)
=0.
\end{align}]]></tex-math></disp-formula></p>
<p>In the twisted gauge, the equations of motion become
<disp-formula id="ptx175-M4-4"><label>(4.4)</label><tex-math notation="LaTeX" id="Equation67"><![CDATA[
\begin{align}
&\left\{
\begin{array}{c}
\overline{\sigma}^\mu\partial_\mu\tilde{u}_{+R}
=\left(+kD_+(c_0)-i\partial_v\right)\tilde{u}_{+L}, \cr
{\sigma}^\mu\partial_\mu\tilde{u}_{+L}
=\left(+kD_-(c_0)-i\partial_v\right)\tilde{u}_{+R},\\
\end{array}
\right.\ \
\left\{
\begin{array}{c}
\overline{\sigma}^\mu\partial_\mu\tilde{u}_{+R}'
=\left(+kD_+(c_0)-i\partial_v\right)\tilde{u}_{+L}' , \cr
{\sigma}^\mu\partial_\mu\tilde{u}_{+L}'
=\left(+kD_-(c_0)-i\partial_v\right)\tilde{u}_{+R}' , \\
\end{array}
\right.\cr
&\left\{
\begin{array}{c}
\overline{\sigma}^\mu\partial_\mu\tilde{u}_{-R}
=\left(-kD_-(c_0)+i\partial_v\right)\tilde{u}_{-L} , \cr
{\sigma}^\mu\partial_\mu\tilde{u}_{-L}
=\left(-kD_+(c_0)+i\partial_v\right)\tilde{u}_{-R} , \\
\end{array}
\right.\ \
\left\{
\begin{array}{c}
\overline{\sigma}^\mu\partial_\mu\tilde{u}_{-R}'
=\left(-kD_-(c_0)+i\partial_v\right)\tilde{u}_{-L}' , \cr
{\sigma}^\mu\partial_\mu\tilde{u}_{-L}'
=\left(-kD_+(c_0)+i\partial_v\right)\tilde{u}_{-R}' , \\
\end{array}
\right.
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M4-5"><label>(4.5)</label><tex-math notation="LaTeX" id="Equation68"><![CDATA[
\begin{align}
D_\pm (c) = \pm \frac{\partial}{\partial z} + \frac{c}{z} ,
\end{align}]]></tex-math></disp-formula>
and the relation between the twisted gauge and the original one is given by
<disp-formula id="ptx175-M4-6"><label>(4.6)</label><tex-math notation="LaTeX" id="Equation69"><![CDATA[
\begin{align}
&\begin{pmatrix} u\\ u' \end{pmatrix}
=\begin{pmatrix}
\cos \frac{1}{2} \theta(z) &-i\sin \frac{1}{2} \theta(z) \cr
-i\sin \frac{1}{2} \theta(z) &\cos \frac{1}{2} \theta(z) \end{pmatrix}
\begin{pmatrix} \tilde u\\ \tilde u' \end{pmatrix}
=:\Omega(z)^{-1}
\begin{pmatrix} \tilde u\\ \tilde u' \end{pmatrix}\!.
\end{align}]]></tex-math></disp-formula></p>
<p>Let us check KK mode expansions for <inline-formula><tex-math notation="LaTeX" id="ImEquation653"><![CDATA[$u_L$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation654"><![CDATA[$u_R$]]></tex-math></inline-formula>. Since <inline-formula><tex-math notation="LaTeX" id="ImEquation655"><![CDATA[$u_L$]]></tex-math></inline-formula> has the parity assignment <inline-formula><tex-math notation="LaTeX" id="ImEquation656"><![CDATA[$\left( {\matrix{{{P_2}} \hfill & {{P_3}} \hfill \cr {{P_0}} \hfill & {{P_1}} \hfill \cr } } \right) = \left( {\matrix{ + \hfill & + \hfill \cr + \hfill & + \hfill \cr } } \right)$]]></tex-math></inline-formula>, we can expand it by using parity even combinations:
<disp-formula id="ptx175-M4-7"><label>(4.7)</label><tex-math notation="LaTeX" id="Equation70"><![CDATA[
\begin{align}
u_L(x,y,v) &=
\sum_{n=0}^{\infty}u_{nL}^C(x,y)f_n^C(v)
+\sum_{n=1}^{\infty}u_{nL}^S(x,y)f_n^S(v) \cr
&
=\frac{1}{\sqrt{2\pi R_6}}
\sum_{n=-\infty}^{\infty}u_{nL}(x,y)e^{inv/R_6},
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M4-8"><label>(4.8)</label><tex-math notation="LaTeX" id="Equation71"><![CDATA[
\begin{align}
&u_{nL}(x,y)=
\begin{cases}
\displaystyle \frac{1}{\sqrt{2}} \big\{ u_{nL}^C(x,y)-i u_{nL}^S(x,y) \big\} &(n>0) \cr
u_{0L}^C(x,y)&(n=0) \cr
\displaystyle \frac{1}{\sqrt{2}} \big\{ u_{-n L}^C(x,y) + i u_{-n L}^S(x,y) \big\} &(n< 0) ,
\end{cases} \cr
&u_{nL}(x,-y)=u_{-nL}(x,y) , u_{nL}(x,L-y)=u_{-nL}(x,L+y) .
\end{align}]]></tex-math></disp-formula></p>
<p>Note that <inline-formula><tex-math notation="LaTeX" id="ImEquation657"><![CDATA[$u_{nL}^C(x,y)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation658"><![CDATA[$u_{nL}^S(x,y)$]]></tex-math></inline-formula> satisfy the same parity property as <inline-formula><tex-math notation="LaTeX" id="ImEquation659"><![CDATA[$\widetilde \Phi_n^C (x,y)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation660"><![CDATA[$\widetilde \Phi_n^S (x,y)$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-13">A.13</xref>), and <inline-formula><tex-math notation="LaTeX" id="ImEquation661"><![CDATA[$f_n^{C} (v)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation662"><![CDATA[$f_n^{S} (v)$]]></tex-math></inline-formula> are defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-14">A.14</xref>). On the other hand, <inline-formula><tex-math notation="LaTeX" id="ImEquation663"><![CDATA[$u_R$]]></tex-math></inline-formula> has the parity assignment <inline-formula><tex-math notation="LaTeX" id="ImEquation664"><![CDATA[$\left( {\matrix{{{P_2}} \hfill & {{P_3}} \hfill \cr {{P_0}} \hfill & {{P_1}} \hfill \cr } } \right) = \left( {\matrix{ - \hfill & - \hfill \cr - \hfill & - \hfill \cr } } \right)$]]></tex-math></inline-formula> so that one can expand it by using parity odd combinations:
<disp-formula id="ptx175-M4-9"><label>(4.9)</label><tex-math notation="LaTeX" id="Equation72"><![CDATA[
\begin{align}
u_R(x,y,v) &=
\sum_{n=0}^{\infty}u_{nR}^S(x,y)f_n^C(v)
+\sum_{n=1}^{\infty}u_{nR}^C(x,y)f_n^S(v) \cr
&=\frac{1}{\sqrt{2\pi R_6}}
\sum_{n=-\infty}^{\infty}u_{nR}(x,y)e^{inv/R_6},
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M4-10"><label>(4.10)</label><tex-math notation="LaTeX" id="Equation73"><![CDATA[
\begin{align}
&u_{nR}(x,y)=
\begin{cases}
\displaystyle \frac{1}{\sqrt{2}} \big\{ u_{nR}^S(x,y)-i u_{nR}^C(x,y) \big\} &(n>0) \cr
u_{0R}^S(x,y)&(n=0) \cr
\displaystyle \frac{1}{\sqrt{2}} \big\{ u_{-n R}^S(x,y) + i u_{-n R}^C(x,y) \big\} &(n< 0) ,
\end{cases} \cr
&u_{nR}(x,-y)=-u_{-nR}(x,y) ,
u_{nR}(x,L-y)=-u_{-nR}(x,L+y) .
\end{align}]]></tex-math></disp-formula></p>
<p>To investigate the physics at energies <inline-formula><tex-math notation="LaTeX" id="ImEquation665"><![CDATA[$\ll m_{{\rm KK}_6}$]]></tex-math></inline-formula>, we keep only sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation666"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes, and replace <inline-formula><tex-math notation="LaTeX" id="ImEquation667"><![CDATA[$u_{L/R}^{(\prime)}(x,y,v)$]]></tex-math></inline-formula> by <inline-formula><tex-math notation="LaTeX" id="ImEquation668"><![CDATA[$u_{0L/R}^{(\prime)}(x,y)$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M4-11"><label>(4.11)</label><tex-math notation="LaTeX" id="Equation74"><![CDATA[
\begin{align}
\left(
\begin{array}{c}
u_L(x,y,v)\\
u_R(x,y,v)\\
u_L'(x,y,v)\\
u_R'(x,y,v)\\
\end{array}
\right)
\ \Rightarrow\
\frac{1}{\sqrt{2\pi R_6}}
\left(
\begin{array}{c}
u_{0L}(x,y)\\
u_{0R}(x,y)\\
u_{0L}'(x,y)\\
u_{0R}'(x,y)\\
\end{array}
\right)\!.
\end{align}]]></tex-math></disp-formula></p>
<p>They have the following BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation669"><![CDATA[$(y_0,y_1)=(0,L_5)$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M4-12"><label>(4.12)</label><tex-math notation="LaTeX" id="Equation75"><![CDATA[
\begin{align}
\left\{
\begin{array}{c}
u_{0L}(x,y_j-y)=+u_{0L}(x,y_j+y),\\
u_{0R}(x,y_j-y)=-u_{0R}(x,y_j+y),\\
u_{0L}'(x,y_j-y)=-u_{0L}'(x,y_j+y),\\
u_{0R}'(x,y_j-y)=+u_{0R}'(x,y_j+y),\\
\end{array}
\right.
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation670"><![CDATA[$j=0,1$]]></tex-math></inline-formula>. The equations of motion in the twisted gauge of Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-6">4.6</xref>) for the <inline-formula><tex-math notation="LaTeX" id="ImEquation671"><![CDATA[$\gamma_{6D}^7=+1$]]></tex-math></inline-formula> fields become
<disp-formula id="ptx175-M4-13"><label>(4.13)</label><tex-math notation="LaTeX" id="Equation76"><![CDATA[
\begin{align}
&\left\{
\begin{array}{c}
\overline{\sigma}^\mu\partial_\mu\tilde{\check u}_{+0R}
=+kD_+(c_0)\tilde{\check u}_{+0L} , \\
{\sigma}^\mu\partial_\mu\tilde{\check u}_{+0L}
=+kD_-(c_0)\tilde{\check u}_{+0R} , \\
\end{array}
\right. \cr
&\left\{
\begin{array}{c}
\overline{\sigma}^\mu\partial_\mu\tilde{\check u}_{+0R}'
=+kD_+(c_0)\tilde{\check u}_{+0L}' , \\
{\sigma}^\mu\partial_\mu\tilde{\check u}_{+0L}'
=+kD_-(c_0)\tilde{\check u}_{+0R}' . \\
\end{array}
\right.
\end{align}]]></tex-math></disp-formula></p>
<p>It follows that <inline-formula><tex-math notation="LaTeX" id="ImEquation672"><![CDATA[$k^2 D_+ D_- \tilde{\check u}_{+0R} = \Box \tilde{\check u}_{+0R} = m^2 \tilde{\check u}_{+0R}$]]></tex-math></inline-formula>, etc. The BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation673"><![CDATA[$z=z_L$]]></tex-math></inline-formula> in the twisted gauge are the same as those in the original gauge:
<disp-formula id="ptx175-M4-14"><label>(4.14)</label><tex-math notation="LaTeX" id="Equation77"><![CDATA[
\begin{align}
\tilde{\check u}_{+0R}=0,\ D_+\tilde{\check u}_{+0L}=0,\
\tilde{\check u}_{+0L}'=0,\ D_-\tilde{\check u}_{+0R}'=0.
\end{align}]]></tex-math></disp-formula></p>
<p>In terms of the basis functions <inline-formula><tex-math notation="LaTeX" id="ImEquation674"><![CDATA[$C_{R/L} (z; \lambda, c)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation675"><![CDATA[$S_{R/L} (z; \lambda, c)$]]></tex-math></inline-formula> given in Appendix B of Ref. [<xref ref-type="bibr" rid="B35">35</xref>], which satisfy
<disp-formula id="ptx175-M4-15"><label>(4.15)</label><tex-math notation="LaTeX" id="Equation78"><![CDATA[
\begin{align}
&D_+ (c) \begin{pmatrix} C_L \cr S_L \end{pmatrix}
= \lambda \begin{pmatrix} S_R \cr C_R \end{pmatrix} \!,
D_- (c) \begin{pmatrix} C_R \cr S_R \end{pmatrix}
= \lambda \begin{pmatrix} S_L \cr C_L \end{pmatrix} \!, \cr
&C_R = C_L = 1 , S_R=S_L = 0 \quad {\rm at }z = z_L ,
\end{align}]]></tex-math></disp-formula>
one can write the mode functions for <inline-formula><tex-math notation="LaTeX" id="ImEquation676"><![CDATA[$\gamma_{6D}^7=+1$]]></tex-math></inline-formula> as
<disp-formula id="ptx175-M4-16"><label>(4.16)</label><tex-math notation="LaTeX" id="Equation79"><![CDATA[
\begin{align}
\begin{pmatrix} \tilde{\check u}_{+0R} \cr \tilde{\check u}_{+0R}' \end{pmatrix}
&=\begin{pmatrix} \alpha_R^u S_R(z;\lambda,c_0)\cr
\alpha_R^{\prime u} C_R(z;\lambda,c_0) \end{pmatrix} f_R (x) ,
\bar \sigma^\mu \partial_\mu f_R(x) = m f_L (x),\cr
\begin{pmatrix} \tilde{\check u}_{+0L}\\ \tilde{\check u}_{+0L}' \end{pmatrix}
&=\begin{pmatrix} \alpha_L^u C_L(z;\lambda,c_0)\cr
\alpha_L^{\prime u} S_L(z;\lambda,c_0)\end{pmatrix} f_L (x) ,
\sigma^\mu \partial_\mu f_L(x) = m f_R (x) .
\end{align}]]></tex-math></disp-formula></p>
<p>The equations of motion in Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-13">4.13</xref>) in the bulk <inline-formula><tex-math notation="LaTeX" id="ImEquation677"><![CDATA[$0<y<L$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation678"><![CDATA[$(1<z<z_L)$]]></tex-math></inline-formula> yield
<disp-formula id="ptx175-M4-17"><label>(4.17)</label><tex-math notation="LaTeX" id="Equation80"><![CDATA[
\begin{align}
\begin{pmatrix} -k\lambda&m \cr m&-k\lambda \end{pmatrix}
\begin{pmatrix} \alpha_R^{u} \cr \alpha_L^{u} \end{pmatrix} = 0 ,\ \mbox{etc.},
\end{align}]]></tex-math></disp-formula>
so that one finds
<disp-formula id="ptx175-M4-18"><label>(4.18)</label><tex-math notation="LaTeX" id="Equation81"><![CDATA[
\begin{align}
m=k\lambda .
\end{align}]]></tex-math></disp-formula></p>
<p>The BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation679"><![CDATA[$z=1$]]></tex-math></inline-formula> in the twisted gauge follow from those in the original gauge:
<disp-formula id="ptx175-M4-19"><label>(4.19)</label><tex-math notation="LaTeX" id="Equation82"><![CDATA[
\begin{align}
u_{+0R}=0\ \Rightarrow\ &
\tilde{u}_{+0R} \cos\frac{\theta_H}{2}-i\tilde{u}_{+0R}'\sin\frac{\theta_H}{2}=0,\\
\end{align}]]></tex-math></disp-formula>
<disp-formula id="ptx175-M4-20"><label>(4.20)</label><tex-math notation="LaTeX" id="Equation83"><![CDATA[
\begin{align}
D_-u_{+0R}'=0\ \Rightarrow\ &
-iD_-\tilde{u}_{+0R}\sin\frac{\theta_H}{2}
+D_-\tilde{u}_{+0R}'\cos\frac{\theta_H}{2}=0.
\end{align}]]></tex-math></disp-formula></p>
<p>We write the above equations in matrix form:
<disp-formula id="ptx175-M4-21"><label>(4.21)</label><tex-math notation="LaTeX" id="Equation84"><![CDATA[
\begin{align}
M_u
\begin{pmatrix} \alpha_R^u \cr \alpha_R^{\prime u} \end{pmatrix}
:=
\begin{pmatrix}
\cos\frac{\theta_H}{2}S_R^0 &-i\sin\frac{\theta_H}{2}C_R^0 \cr
-i\sin\frac{\theta_H}{2}\lambda C_L^0& \cos\frac{\theta_H}{2}\lambda S_L^0 \end{pmatrix}
\begin{pmatrix} \alpha_R^u \cr \alpha_R^{\prime u} \end{pmatrix} =0 ,
\end{align}]]></tex-math></disp-formula>
where we have used shorthand notation such as <inline-formula><tex-math notation="LaTeX" id="ImEquation680"><![CDATA[$S_L^{0}$]]></tex-math></inline-formula> standing for <inline-formula><tex-math notation="LaTeX" id="ImEquation681"><![CDATA[$S_L(1;\lambda,c_0)$]]></tex-math></inline-formula>. In the following we will use the same notation. From <inline-formula><tex-math notation="LaTeX" id="ImEquation682"><![CDATA[$\mbox{det}\, M_u=0$]]></tex-math></inline-formula>, we find the mass formula of the up-type quarks:
<disp-formula id="ptx175-M4-22"><label>(4.22)</label><tex-math notation="LaTeX" id="Equation85"><![CDATA[
\begin{align}
S_L^0 S_R^0+\sin^2\frac{\theta_H}{2}=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>The same formula holds for <inline-formula><tex-math notation="LaTeX" id="ImEquation683"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation684"><![CDATA[$\gamma_{6D}^7=-1$]]></tex-math></inline-formula>. The formula in Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-22">4.22</xref>) is the same as in the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation685"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT in Ref. [<xref ref-type="bibr" rid="B35">35</xref>]. For <inline-formula><tex-math notation="LaTeX" id="ImEquation686"><![CDATA[$\lambda z_L\ll 1$]]></tex-math></inline-formula>, the up-type quark mass spectrum is given by
<disp-formula id="ptx175-M4-23"><label>(4.23)</label><tex-math notation="LaTeX" id="Equation86"><![CDATA[
\begin{align}
m_u\simeq
\left\{
\begin{array}{ll}
k z_L^{-1}\sqrt{1-4c_0^2} \, \sin \frac{1}{2} \theta_H & \mbox{for}\ c_0<\frac{1}{2} , \\
k z_L^{-1/2-c_0}\sqrt{4c_0^2-1} \, \sin \frac{1}{2} \theta_H & \mbox{for}\ c_0>\frac{1}{2} . \\
\end{array}
\right.
\end{align}]]></tex-math></disp-formula></p>
</sec>
<sec id="SEC4.2"><title>4.2. Down-type quark</title>
<p>(ii) <inline-formula><tex-math notation="LaTeX" id="ImEquation687"><![CDATA[$Q_{\rm EM}=- \frac{1}{3}$]]></tex-math></inline-formula>: <inline-formula><tex-math notation="LaTeX" id="ImEquation688"><![CDATA[$d_j,d_j',D_{j+}, D_{j-}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation689"><![CDATA[$(\Psi_{\bf 32}, \Psi_{\bf 11})$]]></tex-math></inline-formula></p>
<p>Consider <inline-formula><tex-math notation="LaTeX" id="ImEquation690"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation691"><![CDATA[$\gamma_{6D}^7 = +1$]]></tex-math></inline-formula>. Parity even modes at <inline-formula><tex-math notation="LaTeX" id="ImEquation692"><![CDATA[$y=0$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation693"><![CDATA[$(P_0, P_2) = (+,+)$]]></tex-math></inline-formula> are <inline-formula><tex-math notation="LaTeX" id="ImEquation694"><![CDATA[$d_{+L}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation695"><![CDATA[$d_{+R}'$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation696"><![CDATA[$D_{+R}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation697"><![CDATA[$D_{-L}$]]></tex-math></inline-formula>. From the action in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-19">2.19</xref>) and the <inline-formula><tex-math notation="LaTeX" id="ImEquation698"><![CDATA[${\cal L}_1^m$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation699"><![CDATA[${\cal L}_3^m$]]></tex-math></inline-formula> terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-28">2.28</xref>), one finds the equations of motion for down-type quarks:
<disp-formula id="ptx175-M4-24"><label>(4.24)</label><tex-math notation="LaTeX" id="Equation87"><![CDATA[
\begin{align}
-i\delta
\begin{pmatrix} d_{+L}^{\dagger}\\ d_{+L}^{\prime\dagger} \end{pmatrix}
:\ &
\big(-k\hat{D}_-(c_{0})+i\partial_v\big)
\begin{pmatrix}\check{d}_{+R}\\ \check{d}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{d}_{+L}\\ \check{d}_{+L}^{\prime} \end{pmatrix}
=0 , \cr
i\delta
\begin{pmatrix} d_{+R}^{\dagger}\\ d_{+R}^{\prime\dagger} \end{pmatrix}
:\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{d}_{+R}\\ \check{d}_{+R}^{\prime} \end{pmatrix}
+\big(-k\hat{D}_+(c_{0})+i\partial_v\big)
\begin{pmatrix} \check{d}_{+L}\\ \check{d}_{+L}^{\prime} \end{pmatrix}
= 2 \mu_1 \, \delta(y)
\begin{pmatrix} 0\\ \check{D}_{-L} \end{pmatrix},\cr
-i\delta D_{+L}^{\dagger}:\ &
\big(-k\hat{D}_-(c_{1})+i\partial_v\big)\check{D}_{+R}
+\sigma^\mu\partial_\mu\check{D}_{+L}
=0 , \cr
i\delta D_{+R}^{\dagger}:\ &
\overline{\sigma}^\mu\partial_\mu\check{D}_{+R}
+\big(-k\hat{D}_+(c_{1})+i\partial_v\big)\check{D}_{+L}
=2\mu_{\bf 11} \, \delta(y) \, \check{D}_{-L} ,\cr
-i\delta D_{-L}^{\dagger}:\ &
\big(k\hat{D}_+(c_{1})-i\partial_v\big)\check{D}_{-R}
+\sigma^\mu\partial_\mu\check{D}_{-L}
=\delta(y)
\left\{2\mu_{\bf 11}\check{D}_{+R}+2\mu_{1}\check{d}_{+R}'\right\}\!,\cr
i\delta D_{-R}^{\dagger}:\ &
\overline{\sigma}^\mu\partial_\mu\check{D}_{-R}
+\big(k\hat{D}_-(c_{1})-i\partial_v\big)\check{D}_{-L}
= 0 .
\end{align}]]></tex-math></disp-formula></p>
<p>For sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation700"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes, the equations of motion reduce to
<disp-formula id="ptx175-M4-25"><label>(4.25)</label><tex-math notation="LaTeX" id="Equation88"><![CDATA[
\begin{align}
\begin{matrix} {\rm (a)}\\ {\rm (b)} \end{matrix}
:\ &
-k\hat{D}_-(c_{0})
\begin{pmatrix} \check{d}_{+R}\\ \check{d}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{d}_{+L}\\ \check{d}_{+L}^{\prime} \end{pmatrix}
=0 ,\cr
\begin{matrix} {\rm (c)}\\ {\rm (d)} \end{matrix}
:\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{d}_{+R}\\ \check{d}_{+R}^{\prime} \end{pmatrix}
-k\hat{D}_+(c_{0})
\begin{pmatrix} \check{d}_{+L}\\ \check{d}_{+L}^{\prime} \end{pmatrix}
=2\mu_1\delta(y)
\begin{pmatrix} 0\\ \check{D}_{-L} \end{pmatrix}\!, \cr
{\rm (e)}:\ &
-k\hat{D}_-(c_{1})\check{D}_{+R}
+\sigma^\mu\partial_\mu\check{D}_{+L}
=0,\cr
{\rm (f)}:\ &
\overline{\sigma}^\mu\partial_\mu\check{D}_{+R}
-k\hat{D}_+(c_{1})\check{D}_{+L}
=2\mu_{\bf 11}\delta(y)\check{D}_{-L},\cr
{\rm (g)}:\ &
k\hat{D}_+(c_{1})\check{D}_{-R}
+\sigma^\mu\partial_\mu\check{D}_{-L}
=2\mu_{\bf 11}\delta(y)\check{D}_{+R}
+2\mu_1\delta(y)\check{d}_{+R}^{\prime},\cr
{\rm (h)}:\ &
\overline{\sigma}^\mu\partial_\mu\check{D}_{-R}
+k\hat{D}_-(c_{1})\check{D}_{-L}
=0.
\end{align}]]></tex-math></disp-formula></p>
<p>To obtain BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation701"><![CDATA[$y=0$]]></tex-math></inline-formula>, we integrate the above (a), (d), (f), and (g) in the vicinity of <inline-formula><tex-math notation="LaTeX" id="ImEquation702"><![CDATA[$y=0$]]></tex-math></inline-formula> for parity-odd fields:
<disp-formula id="ptx175-M4-26"><label>(4.26)</label><tex-math notation="LaTeX" id="Equation89"><![CDATA[
\begin{align}
{\rm (a)}\ \Rightarrow\ &
2\check{d}_{+R}(x,\epsilon)=0,\cr
{\rm (d)}\ \Rightarrow\ &
-2\check{d}_{+L}^{\prime}(x,\epsilon)=
2\mu_1\check{D}_{-L}(x,0),\cr
{\rm (f)}\ \Rightarrow\ &
-2\check{D}_{+L}(x,\epsilon)=
2\mu_{\bf 11}\check{D}_{-L}(x,0),\cr
{\rm (g)}\ \Rightarrow\ &
2\check{D}_{-R}(x,\epsilon)=
2\mu_{\bf 11}\check{D}_{+R}(x,0)
+2\mu_1\check{d}_{+R}^{\prime}(x,0).
\end{align}]]></tex-math></disp-formula></p>
<p>For parity-even fields, we evaluate the equations of motion at <inline-formula><tex-math notation="LaTeX" id="ImEquation703"><![CDATA[$y=+\epsilon$]]></tex-math></inline-formula> by using the above conditions:
<disp-formula id="ptx175-M4-27"><label>(4.27)</label><tex-math notation="LaTeX" id="Equation90"><![CDATA[
\begin{align}
{\rm (c)}\ \Rightarrow\ &
\hat{D}_+\check{d}_{+L}(x,\epsilon)=0,\cr
{\rm (b)}\ \Rightarrow\ &
-k\hat{D}_-\check{d}_{+R}^{\prime}
+\mu_1 k D_{+}\check{D}_{-R}
=0,\cr
{\rm (e)}\ \Rightarrow\ &
-k{D}_-\check{D}_{+R}(x,\epsilon)
+\mu_{\bf 11}kD_{+}\check{D}_{-R}=0,\cr
{\rm (h)}\ \Rightarrow\ &
k{D}_-\check{D}_{-L}
+\mu_{\bf 11}kD_{+}\check{D}_{+L}
+\mu_1 k\hat{D}_{+}\check{d}_{+L}^{\prime}
=0,
\end{align}]]></tex-math></disp-formula>
where we used the equations of motion (d) and (g) at <inline-formula><tex-math notation="LaTeX" id="ImEquation704"><![CDATA[$y=+\epsilon$]]></tex-math></inline-formula>.</p>
<p>We recall that in the twisted gauge all fields obey free-field equations in the bulk. Their eigenmodes are determined by the BCs on the IR brane. The mass spectra can be fixed by the BCs at the UV brane. In the twisted gauge the mode functions are given by
<disp-formula id="ptx175-M4-28"><label>(4.28)</label><tex-math notation="LaTeX" id="Equation91"><![CDATA[
\begin{align}
&\begin{pmatrix}
\tilde{\check{d}}_{+R}\cr \tilde{\check{d}}_{+R}^{\prime}\cr
\tilde{\check{D}}_{+R}\cr \tilde{\check{D}}_{-R} \end{pmatrix}
= \begin{pmatrix}
\alpha_R^{d}S_R(z;\lambda,c_0) \cr
\alpha_R^{d'}C_R(z;\lambda,c_0) \cr
\alpha_R^{D_+}C_R(z;\lambda,c_1)\cr
\alpha_R^{D_-}S_L(z;\lambda,c_1) \end{pmatrix} f_R(x) ,\cr
&\begin{pmatrix}
\tilde{\check{d}}_{+L}\cr
\tilde{\check{d}}_{+L}^{\prime}\cr
\tilde{\check{D}}_{+L}\cr
\tilde{\check{D}}_{-L} \end{pmatrix}
=\begin{pmatrix}
\alpha_L^{d}C_L(z;\lambda,c_0)\cr
\alpha_L^{d'}S_L(z;\lambda,c_0)\cr
\alpha_L^{D_+}S_L(z;\lambda,c_1)\cr
\alpha_L^{D_-}C_R(z;\lambda,c_1) \end{pmatrix} f_L(x) ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation705"><![CDATA[$f_R (x), f_L (x)$]]></tex-math></inline-formula> satisfy the relations in Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-16">4.16</xref>) and <inline-formula><tex-math notation="LaTeX" id="ImEquation706"><![CDATA[$m= k\lambda$]]></tex-math></inline-formula>. The BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation707"><![CDATA[$z=1^+$]]></tex-math></inline-formula> in the twisted gauge are converted to
<disp-formula id="ptx175-M4-29"><label>(4.29)</label><tex-math notation="LaTeX" id="Equation92"><![CDATA[
\begin{align}
&K\begin{pmatrix}
\alpha_R^{d}\cr
\alpha_R^{d'}\cr
\alpha_R^{D_+}\cr
\alpha_R^{D_-} \end{pmatrix} = 0 , \cr
&K =
\begin{pmatrix}
\cos\frac{\theta_H}{2}S_R^0&-i\sin\frac{\theta_H}{2}C_R^0&0&0\cr -i\sin\frac{\theta_H}{2}\lambda
C_L^0&\cos\frac{\theta_H}{2}\lambda S_L^0& 0&-\mu_1\lambda C_R^1\cr 0&0&\lambda S_L^1&-\mu_{\bf 11}\lambda C_R^1\cr
i\mu_1\sin\frac{\theta_H}{2}S_R^0&-\mu_1\cos\frac{\theta_H}{2}C_R^0&
-\mu_{\bf 11}C_R^1&S_L^1 \end{pmatrix} \!.
\end{align}]]></tex-math></disp-formula></p>
<p>From <inline-formula><tex-math notation="LaTeX" id="ImEquation708"><![CDATA[$\mbox{det}\,K=0$]]></tex-math></inline-formula>, we find the mass spectrum formula for the down-type quarks:
<disp-formula id="ptx175-M4-30"><label>(4.30)</label><tex-math notation="LaTeX" id="Equation93"><![CDATA[
\begin{align}
S_L^0 S_R^0+\sin^2\frac{\theta_H}{2}=
-\frac{\mu_1^2 S_R^0 C_R^0 S_L^1 C_R^1}
{\mu_{\bf 11}^2(C_R^1)^2-(S_L^1)^2}.
\end{align}]]></tex-math></disp-formula></p>
<p>The same formula is obtained for <inline-formula><tex-math notation="LaTeX" id="ImEquation709"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation710"><![CDATA[$\gamma_{6D}^7 = -1$]]></tex-math></inline-formula>. For <inline-formula><tex-math notation="LaTeX" id="ImEquation711"><![CDATA[$\lambda z_L\ll 1$]]></tex-math></inline-formula>, the down-type quark mass spectrum is given by
<disp-formula id="ptx175-M4-31"><label>(4.31)</label><tex-math notation="LaTeX" id="Equation94"><![CDATA[
\begin{align}
m_d \simeq
\begin{cases}
k z_L^{c_0-c_1-1}
\displaystyle \frac{\mu_{\bf 11}}{\mu_1}
\sqrt{(1-2c_0)(1+2c_1)} \, \sin \displaystyle \frac{\theta_H}{2}&
\mbox{for}\ c_0<\frac{1}{2} , \cr
k z_L^{-c_1-1/2}
\displaystyle \frac{\mu_{\bf 11}}{\mu_1}
\sqrt{(2c_0-1)(2c_1+1)} \, \sin \displaystyle \frac{\theta_H}{2}&
\mbox{for}\ c_0> \frac{1}{2} . \end{cases}
\end{align}]]></tex-math></disp-formula></p>
<p>Combining Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-23">4.23</xref>) and (<xref ref-type="disp-formula" rid="ptx175-M4-31">4.31</xref>), one finds
<disp-formula id="ptx175-M4-32"><label>(4.32)</label><tex-math notation="LaTeX" id="Equation95"><![CDATA[
\begin{align}
\frac{m_d}{m_u} = z_L^{c_0 - c_1} \, \frac{\mu_{\bf 11}}{\mu_1}
\sqrt{\frac{1 + 2 c_1}{1+ 2 c_0}} .
\end{align}]]></tex-math></disp-formula></p>
</sec>
<sec id="SEC4.3"><title>4.3. Charged lepton</title>
<p>(iii) <inline-formula><tex-math notation="LaTeX" id="ImEquation712"><![CDATA[$Q_{\rm EM}=-1$]]></tex-math></inline-formula>: <inline-formula><tex-math notation="LaTeX" id="ImEquation713"><![CDATA[$e,e',E_+^{\prime},E_-^{\prime}$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation714"><![CDATA[$(\Psi_{\bf 32}, \Psi_{\bf 11}')$]]></tex-math></inline-formula></p>
<p>Parity even modes at <inline-formula><tex-math notation="LaTeX" id="ImEquation715"><![CDATA[$y=0$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation716"><![CDATA[$(P_0, P_2) = (+,+)$]]></tex-math></inline-formula> are <inline-formula><tex-math notation="LaTeX" id="ImEquation717"><![CDATA[$e_{+L}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation718"><![CDATA[$e_{+R}'$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation719"><![CDATA[$E_{+L}'$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation720"><![CDATA[$E_{-R}'$]]></tex-math></inline-formula>. From the action in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-19">2.19</xref>) and the <inline-formula><tex-math notation="LaTeX" id="ImEquation721"><![CDATA[${\cal L}_5^m$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation722"><![CDATA[${\cal L}_6^m$]]></tex-math></inline-formula> terms in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-28">2.28</xref>), one finds the equations of motion for charged leptons:
<disp-formula id="ptx175-M4-33"><label>(4.33)</label><tex-math notation="LaTeX" id="Equation96"><![CDATA[
\begin{align}
-i\delta
\begin{pmatrix} e_{+L}^{\dagger}\\ e_{+L}^{\prime\dagger} \end{pmatrix}
:\ &
\big(-k\hat{D}_-(c_{0})+i\partial_v\big)
\begin{pmatrix}\check{e}_{+R}\\ \check{e}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{e}_{+L}\\ \check{e}_{+L}^{\prime} \end{pmatrix}
= 2 i \tilde\mu_2 \, \delta(y)
\begin{pmatrix} \check{E}_{-R}^{\prime} \\ 0 \end{pmatrix} \!, \cr
i\delta
\begin{pmatrix} e_{+R}^{\dagger}\\ e_{+R}^{\prime\dagger} \end{pmatrix}
:\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{e}_{+R}\\ \check{e}_{+R}^{\prime} \end{pmatrix}
+\big(-k\hat{D}_+(c_{0})+i\partial_v\big)
\begin{pmatrix} \check{e}_{+L}\\ \check{e}_{+L}^{\prime} \end{pmatrix}
= 0 , \cr
-i\delta E_{+L}^{\prime \dagger}:\ &
\big(-k\hat{D}_-(c_{2})+i\partial_v\big)\check{E}_{+R}'
+\sigma^\mu\partial_\mu\check{E}_{+L}'
= 2\mu_{\bf 11}^{\prime}\delta(y)\check{E}_{-R}^{\prime} , \cr
i\delta E_{+R}^{\prime \dagger}:\ &
\overline{\sigma}^\mu\partial_\mu\check{E}_{+R}'
+\big(-k\hat{D}_+(c_{2})+i\partial_v\big)\check{E}_{+L}' = 0 , \cr
-i\delta E_{-L}^{\prime \dagger}:\ &
\big(k\hat{D}_+(c_{2})-i\partial_v\big)\check{E}_{-R}'
+\sigma^\mu\partial_\mu\check{E}_{-L}' = 0 , \cr
i\delta E_{-R}^{\prime \dagger}:\ &
\overline{\sigma}^\mu\partial_\mu\check{E}_{-R}'
+\big(k\hat{D}_-(c_{2})-i\partial_v\big)\check{E}_{-L}'
=\delta(y)\left\{
-2i\widetilde{\mu}_2 \check{e}_{+L}
+2\mu_{\bf 11}^{\prime}\check{E}_{+L}^{\prime} \right\}.
\end{align}]]></tex-math></disp-formula></p>
<p>For sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation723"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes, the equations of motion for charged leptons reduce to
<disp-formula id="ptx175-M4-34"><label>(4.34)</label><tex-math notation="LaTeX" id="Equation97"><![CDATA[
\begin{align}
\begin{matrix} {\rm (a)}\\ {\rm (b)} \end{matrix}
:\ &
-k\hat{D}_-(c_{0})
\begin{pmatrix}\check{e}_{+R}\\ \check{e}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{e}_{+L}\\ \check{e}_{+L}^{\prime} \end{pmatrix}
= 2 i \tilde\mu_2 \, \delta(y)
\begin{pmatrix} \check{E}_{-R}^{\prime} \\ 0 \end{pmatrix} \!, \cr
\begin{matrix} {\rm (c)}\\ {\rm (d)} \end{matrix}
:\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{e}_{+R}\\ \check{e}_{+R}^{\prime} \end{pmatrix}
-k\hat{D}_+(c_{0})
\begin{pmatrix} \check{e}_{+L}\\ \check{e}_{+L}^{\prime} \end{pmatrix}
= 0 , \cr
{\rm (e)}:\ &
-k\hat{D}_-(c_{2}) \check{E}_{+R}'
+\sigma^\mu\partial_\mu\check{E}_{+L}'
= 2\mu_{\bf 11}^{\prime}\delta(y)\check{E}_{-R}^{\prime} , \cr
{\rm (f)}:\ &
\overline{\sigma}^\mu\partial_\mu\check{E}_{+R}'
-k\hat{D}_+(c_{2}) \check{E}_{+L}' = 0 , \cr
{\rm (g)}:\ &
k\hat{D}_+(c_{2}) \check{E}_{-R}'
+\sigma^\mu\partial_\mu\check{E}_{-L}' = 0 , \cr
{\rm (h)}:\ &
\overline{\sigma}^\mu\partial_\mu\check{E}_{-R}'
+ k\hat{D}_-(c_{2}) \check{E}_{-L}'
=\delta(y)\left\{
-2i\widetilde{\mu}_2 \check{e}_{+L}
+2\mu_{\bf 11}^{\prime}\check{E}_{+L}^{\prime} \right\}.
\end{align}]]></tex-math></disp-formula></p>
<p>To obtain boundary conditions at <inline-formula><tex-math notation="LaTeX" id="ImEquation724"><![CDATA[$y=0$]]></tex-math></inline-formula>, we integrate the above (a), (d), (e), (h) in the vicinity of <inline-formula><tex-math notation="LaTeX" id="ImEquation725"><![CDATA[$y=0$]]></tex-math></inline-formula> for parity-odd fields:
<disp-formula id="ptx175-M4-35"><label>(4.35)</label><tex-math notation="LaTeX" id="Equation98"><![CDATA[
\begin{align}
{\rm (a)}\ \Rightarrow\ &
2\check{e}_{+R}(x,\epsilon)=2i\widetilde{\mu}_2\check{E}_{-R}^{\prime}(x,0),\cr
{\rm (d)}\ \Rightarrow\ &
-2\check{e}_{+L}^{\prime}(x,\epsilon)=
0,\cr
{\rm (e)}\ \Rightarrow\ &
2\check{E}_{+R}^{\prime}(x,\epsilon)=
2\mu_{\bf 11}^{\prime}\check{E}_{-R}^{\prime}(x,0),\cr
{\rm (h)}\ \Rightarrow\ &
-2\check{E}_{-L}^{\prime}(x,\epsilon)=
-i2\widetilde{\mu}_{2}\check{e}_{+L}(x,0)
+2\mu_{\bf 11}^{\prime}\check{E}_{+L}^{\prime}(x,0).
\end{align}]]></tex-math></disp-formula></p>
<p>For parity-even fields, we calculate the equations of motion at <inline-formula><tex-math notation="LaTeX" id="ImEquation726"><![CDATA[$y=+\epsilon$]]></tex-math></inline-formula> by using the above conditions:
<disp-formula id="ptx175-M4-36"><label>(4.36)</label><tex-math notation="LaTeX" id="Equation99"><![CDATA[
\begin{align}
{\rm (b)}\ \Rightarrow\ &
\hat{D}_-\check{e}_{+R}^{\prime}(x,\epsilon)=
0,\cr
{\rm (c)}\ \Rightarrow\ &
\hat{D}_+\check{e}_{+L}
+i\widetilde{\mu}_2D_{-}\check{E}_{-L}^{\prime}
=0,\cr
{\rm (f)}\ \Rightarrow\ &
D_+\check{E}_{+L}^{\prime}(x,\epsilon)
+\mu_{\bf 11}^{\prime}D_{-}\check{E}_{-L}^{\prime}=0,\cr
{\rm (g)}\ \Rightarrow\ &
{D}_+\check{E}_{-R}^{\prime}
+i\widetilde{\mu}_{2}\hat{D}_{-}\check{e}_{+R}
-\mu_{\bf 11}^{\prime}{D}_{-}\check{E}_{+R}^{\prime}
=0,
\end{align}]]></tex-math></disp-formula>
where we used the equations of motion (d) and (h) at <inline-formula><tex-math notation="LaTeX" id="ImEquation727"><![CDATA[$y=+\epsilon$]]></tex-math></inline-formula>.</p>
<p>By using the BCs on the IR brane, the mode functions of charged leptons in the twisted gauge are given by
<disp-formula id="ptx175-M4-37"><label>(4.37)</label><tex-math notation="LaTeX" id="Equation100"><![CDATA[
\begin{align}
&\begin{pmatrix}
\tilde{\check{e}}_{+R}\cr \tilde{\check{e}}_{+R}^{\prime}\cr
\tilde{\check{E}}_{+R}'\cr \tilde{\check{E}}_{-R}' \end{pmatrix}
= \begin{pmatrix}
\alpha_R^{e}S_R(z;\lambda,c_0) \cr
\alpha_R^{e'}C_R(z;\lambda,c_0) \cr
\alpha_R^{E_+'}S_R(z;\lambda,c_2)\cr
\alpha_R^{E_-'}C_L(z;\lambda,c_2) \end{pmatrix} f_R(x) ,\cr
&\begin{pmatrix}
\tilde{\check{e}}_{+L}\cr
\tilde{\check{e}}_{+L}^{\prime}\cr
\tilde{\check{E}}_{+L}'\cr
\tilde{\check{E}}_{-L}' \end{pmatrix}
=\begin{pmatrix}
\alpha_L^{e}C_L(z;\lambda,c_0)\cr
\alpha_L^{e'}S_L(z;\lambda,c_0)\cr
\alpha_L^{E_+'}C_L(z;\lambda,c_2)\cr
\alpha_L^{E_-'}S_R(z;\lambda,c_2) \end{pmatrix} f_L(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>As in the case of down-type quarks, the BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation728"><![CDATA[$z=1^+$]]></tex-math></inline-formula> in the twisted gauge are converted to
<disp-formula id="ptx175-M4-38"><label>(4.38)</label><tex-math notation="LaTeX" id="Equation101"><![CDATA[
\begin{align}
&K \begin{pmatrix}
\alpha_R^{e} \cr
\alpha_R^{e'} \cr
\alpha_R^{E_+'}\cr
\alpha_R^{E_-'} \end{pmatrix} = 0 , \cr
&K = \begin{pmatrix}
\cos\frac{\theta_H}{2}S_R^0 &-i\sin\frac{\theta_H}{2}C_R^0 &0
&-i\widetilde{\mu}_2C_L^2\cr
-i\sin\frac{\theta_H}{2}\lambda C_L^0 &\cos\frac{\theta_H}{2}\lambda S_L^0
&0 &0 \cr
0 &0 &S_R^2 &-\mu_{\bf 11}^{\prime}C_L^2 \cr
i\widetilde{\mu}_2\cos\frac{\theta_H}{2}\lambda C_L^0 &
\widetilde{\mu}_2\sin\frac{\theta_H}{2}\lambda S_L^0 &
-\mu_{\bf 11}^{\prime}\lambda C_L^2 &\lambda S_R^2
\end{pmatrix} .
\end{align}]]></tex-math></disp-formula></p>
<p>From <inline-formula><tex-math notation="LaTeX" id="ImEquation729"><![CDATA[$\mbox{det}\,K=0$]]></tex-math></inline-formula>, we find the mass spectrum formula for the charged leptons:
<disp-formula id="ptx175-M4-39"><label>(4.39)</label><tex-math notation="LaTeX" id="Equation102"><![CDATA[
\begin{align}
S_L^0 S_R^0+\sin^2\frac{\theta_H}{2}=
-\frac{\widetilde{\mu}_2^2 S_L^0 C_L^0 S_R^2 C_L^2}
{\mu_{\bf 11}^{\prime 2}(C_L^2)^2-(S_R^2)^2}.
\end{align}]]></tex-math></disp-formula></p>
<p>The mass spectrum of charged leptons, for which <inline-formula><tex-math notation="LaTeX" id="ImEquation730"><![CDATA[$\lambda z_L\ll 1$]]></tex-math></inline-formula>, is given by
<disp-formula id="ptx175-M4-40"><label>(4.40)</label><tex-math notation="LaTeX" id="Equation103"><![CDATA[
\begin{align}
m_e\simeq
\begin{cases}
k z_L^{-1+c_2-c_0} \,
\displaystyle \frac{\mu_{\bf 11}^{\prime}}{\widetilde{\mu}_2} \,
\sqrt{(2c_0+1)(1-2c_2)} \,
\sin \displaystyle \frac{\theta_H}{2}
&\mbox{for}\ c_2<\frac{1}{2}, \cr
k z_L^{-\frac{1}{2}-c_0} \,
\displaystyle \frac{\mu_{\bf 11}^{\prime}}{\widetilde{\mu}_2} \,
\sqrt{(2c_0+1)(2c_2-1)} \,
\sin \displaystyle \frac{\theta_H}{2}
&\mbox{for}\ c_2>\frac{1}{2}.
\end{cases}
\end{align}]]></tex-math></disp-formula></p>
<p>Here we have made use of <inline-formula><tex-math notation="LaTeX" id="ImEquation731"><![CDATA[$(m_t/m_\tau)^2, (m_c/m_\mu)^2, (m_u/m_e)^2 \gg 1$]]></tex-math></inline-formula>, which assures <inline-formula><tex-math notation="LaTeX" id="ImEquation732"><![CDATA[$|S_L^0 S_R^0 | \ll \sin^2 \frac{1}{2} \theta_H$]]></tex-math></inline-formula>, and have assumed that <inline-formula><tex-math notation="LaTeX" id="ImEquation733"><![CDATA[$\lambda^2 \ll\mu_{\bf 11}'^2$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation734"><![CDATA[$c_2 > \frac{1}{2}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation735"><![CDATA[$\lambda^2\ll(\mu_{\bf 11}' z_L^{2 c_2-1})^2$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation736"><![CDATA[$c_2 < \frac{1}{2}$]]></tex-math></inline-formula> so that <inline-formula><tex-math notation="LaTeX" id="ImEquation737"><![CDATA[$\mu_{\bf 11}^{\prime 2}(C_L^2)^2 \gg (S_R^2)^2$]]></tex-math></inline-formula>. Combining Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-23">4.23</xref>) and (<xref ref-type="disp-formula" rid="ptx175-M4-40">4.40</xref>), one finds
<disp-formula id="ptx175-M4-41"><label>(4.41)</label><tex-math notation="LaTeX" id="Equation104"><![CDATA[
\begin{align}
&\frac{m_e}{m_u} =\begin{cases}
z_L^{c_2 - c_0}\, \displaystyle \frac{\mu_{\bf 11}'}{\widetilde{\mu}_2}\, \sqrt{\frac{1 - 2 c_2}{1-2 c_0}}
&\mbox{for}\ c_0 <\frac{1}{2},~ c_2 < \frac{1}{2} , \cr
z_L^{\frac{1}{2} - c_0}\, \displaystyle \frac{\mu_{\bf 11}'}{\widetilde{\mu}_2}\,
\sqrt{\frac{2 c_2 - 1}{1-2 c_0}}
&\mbox{for}\ c_0 <\frac{1}{2},~ c_2 > \frac{1}{2} , \cr
z_L^{c_2 - \frac{1}{2}}\, \displaystyle \frac{\mu_{\bf 11}'}{\widetilde{\mu}_2}\,
\sqrt{\frac{1 - 2 c_2}{2 c_0 - 1}}
&\mbox{for}\ c_0 > \frac{1}{2},~ c_2 < \frac{1}{2} , \cr
\displaystyle \frac{\mu_{\bf 11}'}{\widetilde{\mu}_2}\,
\sqrt{\frac{2 c_2 - 1}{2 c_0 - 1}}
&\mbox{for}\ c_0 >\frac{1}{2},~ c_2 > \frac{1}{2}.
\end{cases}
\end{align}]]></tex-math></disp-formula></p>
<p>It will be seen below that in a typical example for the third generation <inline-formula><tex-math notation="LaTeX" id="ImEquation738"><![CDATA[$c_2=-0.7$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation739"><![CDATA[$\mu^{'}_{\bf 11}C_{L}^2/S_{R}^2\sim 2.6$]]></tex-math></inline-formula>.</p>
<p>One comment is in order about the masses of exotic charged leptons <inline-formula><tex-math notation="LaTeX" id="ImEquation740"><![CDATA[$\hat e, \hat e', \hat E_\pm'$]]></tex-math></inline-formula> with charge <inline-formula><tex-math notation="LaTeX" id="ImEquation741"><![CDATA[$Q_{\rm EM} = +1$]]></tex-math></inline-formula>. No zero modes exist for <inline-formula><tex-math notation="LaTeX" id="ImEquation742"><![CDATA[$\hat e$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation743"><![CDATA[$\hat e'$]]></tex-math></inline-formula>, and all modes have masses larger than <inline-formula><tex-math notation="LaTeX" id="ImEquation744"><![CDATA[$\frac{1}{2} m_{{\rm KK}_6}$]]></tex-math></inline-formula>. On the other hand <inline-formula><tex-math notation="LaTeX" id="ImEquation745"><![CDATA[$\hat E_\pm'$]]></tex-math></inline-formula> have zero modes. They acquire masses through the Hosotani mechanism and brane interactions. We suppose that <inline-formula><tex-math notation="LaTeX" id="ImEquation746"><![CDATA[$c_2 < \frac{1}{2}$]]></tex-math></inline-formula> and/or <inline-formula><tex-math notation="LaTeX" id="ImEquation747"><![CDATA[$\mu_{\bf 11}'$]]></tex-math></inline-formula> is sufficiently large that their lightest masses are <inline-formula><tex-math notation="LaTeX" id="ImEquation748"><![CDATA[$O(m_{{\rm KK}_5})$]]></tex-math></inline-formula>.</p>
</sec>
<sec id="SEC4.4"><title>4.4. Neutrino</title>
<p>(iv) <inline-formula><tex-math notation="LaTeX" id="ImEquation749"><![CDATA[$Q_{\rm EM}=0$]]></tex-math></inline-formula>: <inline-formula><tex-math notation="LaTeX" id="ImEquation750"><![CDATA[$\nu,\nu',N_\pm', \hat N_\pm', S_\pm' , \eta_-$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation751"><![CDATA[$(\Psi_{\bf 32}, \Psi_{\bf 11}^{\prime}, \chi_{\bf 1})$]]></tex-math></inline-formula></p>
<p>The 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation752"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor and vector bulk fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation753"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation754"><![CDATA[$\Psi_{\bf 11}'$]]></tex-math></inline-formula> and a 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation755"><![CDATA[$SO(11)$]]></tex-math></inline-formula> singlet symplectic Majorana brane fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation756"><![CDATA[$\chi$]]></tex-math></inline-formula> appear in this sector. From the action in Eqs. (<xref ref-type="disp-formula" rid="ptx175-M2-19">2.19</xref>) and <inline-formula><tex-math notation="LaTeX" id="ImEquation757"><![CDATA[${\cal L}_5^m, {\cal L}_6^m, {\cal L}_7^m$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-28">2.28</xref>), the equations of motion for the bulk fermions become
<disp-formula id="ptx175-M4-42"><label>(4.42)</label><tex-math notation="LaTeX" id="Equation105"><![CDATA[
\begin{align}
-i\delta
\begin{pmatrix} \nu_{+L}^{\dagger}\\ \nu_{+L}^{\prime\dagger} \end{pmatrix} :\ &
\big( -k\hat{D}_-(c_0)+i\partial_v \big)
\begin{pmatrix} \check{\nu}_{+R}\\ \check{\nu}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{\nu}_{+L}\\ \check{\nu}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 2i\widetilde{\mu}_2\check{N}_{-R}^{\prime}\cr
(m_B/\sqrt{k}) \, \xi_- \end{pmatrix} \!, \cr
i\delta
\begin{pmatrix} \nu_{+R}^{\dagger}\\ \nu_{+R}^{\prime\dagger} \end{pmatrix} :\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{\nu}_{+R}\\ \check{\nu}_{+R}^{\prime} \end{pmatrix}
+ \big( -k\hat{D}_+(c_{0})+i\partial_v\big)
\begin{pmatrix} \check{\nu}_{+L}\\ \check{\nu}_{+L}^{\prime} \end{pmatrix} \cr
&\hspace {6cm}
=\delta(y)
\begin{pmatrix} 0\\ -\sqrt{2}\widetilde{\mu}_2\check{S}_{-L}^{\prime}
+ (m_B/\sqrt{k}) \, \eta_- \end{pmatrix}, \cr
-i\delta
\begin{pmatrix} \hat{N}_{+L}^{\prime\dagger}\\ N_{+L}^{\prime\dagger}\\
S_{+L}^{\prime\dagger} \end{pmatrix} :\ &
\big(-k\hat{D}_-(c_{2})+i\partial_v\big)
\begin{pmatrix} \check{\hat{N}}_{+R}^{\prime}\\ \check{N}_{+R}^{\prime}\\
\check{S}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{+L}^{\prime}\\ \check{N}_{+L}^{\prime}\\
\check{S}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 2{\mu}_{\bf 11}^{\prime}\check{\hat{N}}_{-R}^{\prime}\\
2{\mu}_{\bf 11}^{\prime}\check{N}_{-R}^{\prime} \cr 0 \end{pmatrix},\cr
i\delta
\begin{pmatrix} \hat{N}_{+R}^{\prime\dagger}\\ N_{+R}^{\prime\dagger}\\
S_{+R}^{\prime\dagger} \end{pmatrix} :\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{+R}^{\prime}\\ \check{N}_{+R}^{\prime}\\
\check{S}_{+R}^{\prime} \end{pmatrix}
+\big(-k\hat{D}_+(c_{2})+i\partial_v\big)
\begin{pmatrix} \check{\hat{N}}_{+L}^{\prime}\\ \check{N}_{+L}^{\prime}\\
\check{S}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 0 \cr 0 \cr 2{\mu}_{\bf 11}^{\prime}\check{S}_{-L}^{\prime}\end{pmatrix},\cr
-i\delta
\begin{pmatrix} \hat{N}_{-L}^{\prime\dagger}\\ N_{-L}^{\prime\dagger}\\
S_{-L}^{\prime\dagger} \end{pmatrix} :\ &
\big( k\hat{D}_+ (c_{2}) - i\partial_v\big)
\begin{pmatrix} \check{\hat{N}}_{-R}^{\prime}\\ \check{N}_{-R}^{\prime}\\
\check{S}_{-R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{-L}^{\prime}\\ \check{N}_{-L}^{\prime}\\
\check{S}_{-L}^{\prime} \end{pmatrix} \cr
&\hspace {6cm}
=\delta(y)
\begin{pmatrix} 0\\ 0\\
-\sqrt{2}\widetilde{\mu}_{2} \check{\nu}_{+R}^{\prime}
+2\mu_{\bf 11}^{\prime}\check{S}_{+R}^{\prime} \end{pmatrix},\cr
i\delta
\begin{pmatrix} \hat{N}_{-R}^{\prime\dagger}\\ N_{-R}^{\prime\dagger}\\
S_{-R}^{\prime\dagger} \end{pmatrix} :\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{-R}^{\prime}\\ \check{N}_{-R}^{\prime}\\
\check{S}_{-R}^{\prime} \end{pmatrix}
+\big( k\hat{D}_- (c_{2}) - i\partial_v\big)
\begin{pmatrix} \check{\hat{N}}_{-L}^{\prime}\\ \check{N}_{-L}^{\prime}\\
\check{S}_{-L}^{\prime} \end{pmatrix} \cr
&\hspace {6cm}
=\delta(y)
\begin{pmatrix}
2\mu_{\bf 11}^{\prime}\check{\hat{N}}_{+L}^{\prime}\\
-2i\widetilde{\mu}_{2} \check{\nu}_{+L}
+2\mu_{\bf 11}^{\prime}\check{N}_{+L}^{\prime}\\
0 \end{pmatrix}.
\end{align}]]></tex-math></disp-formula></p>
<p>The equations for the 5D brane symplectic Majorana fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation758"><![CDATA[$\chi_{\bf 1}$]]></tex-math></inline-formula> are
<disp-formula id="ptx175-M4-43"><label>(4.43)</label><tex-math notation="LaTeX" id="Equation106"><![CDATA[
\begin{align}
-i \delta \eta_-^\dagger :\ &
\sigma^\mu\partial_\mu\eta_-i\partial_v\xi_-
- \frac{m_B}{\sqrt{k}} \, \nu_{+R}^{\prime}-M\eta_-^C =0 , \cr
i \delta \xi_-^\dagger :\ &
\overline{\sigma}^\mu\partial_\mu\xi_-i\partial_v\eta_-
- \frac{m_B}{\sqrt{k}} \, \nu_{+L}^{\prime}-M\xi_-^C =0 .
\end{align}]]></tex-math></disp-formula></p>
<p>Note that <inline-formula><tex-math notation="LaTeX" id="ImEquation759"><![CDATA[$\eta_-$]]></tex-math></inline-formula> has a zero mode but <inline-formula><tex-math notation="LaTeX" id="ImEquation760"><![CDATA[$\xi_-$]]></tex-math></inline-formula> has no zero mode in the sixth-dimensional direction.</p>
<p>For sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation761"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes, the equations of motion become
<disp-formula id="ptx175-M4-44"><label>(4.44)</label><tex-math notation="LaTeX" id="Equation107"><![CDATA[
\begin{align}
\begin{matrix} {\rm (a)} \cr {\rm (b)} \end{matrix} :\ &
-k\hat{D}_-(c_0)
\begin{pmatrix} \check{\nu}_{+R}\\ \check{\nu}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{\nu}_{+L}\\ \check{\nu}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 2i\widetilde{\mu}_2\check{N}_{-R}^{\prime}\\ 0 \end{pmatrix} \!, \cr
\begin{matrix} {\rm (c)} \cr {\rm (d)} \end{matrix} :\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{\nu}_{+R}\\ \check{\nu}_{+R}^{\prime} \end{pmatrix}
-k\hat{D}_+(c_{0})
\begin{pmatrix} \check{\nu}_{+L}\\ \check{\nu}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 0\\ -\sqrt{2}\widetilde{\mu}_2\check{S}_{-L}^{\prime}
+ (m_B/\sqrt{k}) \, \eta_- \end{pmatrix}, \cr
\begin{matrix} {\rm (e)} \cr {\rm (f)} \cr {\rm (g)} \end{matrix} :\ &
-k\hat{D}_-(c_{2})
\begin{pmatrix} \check{\hat{N}}_{+R}^{\prime}\\ \check{N}_{+R}^{\prime}\\
\check{S}_{+R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{+L}^{\prime}\\ \check{N}_{+L}^{\prime}\\
\check{S}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 2{\mu}_{\bf 11}^{\prime}\check{\hat{N}}_{-R}^{\prime}\\
2{\mu}_{\bf 11}^{\prime}\check{N}_{-R}^{\prime} \cr 0 \end{pmatrix},\cr
\begin{matrix} {\rm (h)} \cr {\rm (i)} \cr {\rm (j)} \end{matrix} :\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{+R}^{\prime}\\ \check{N}_{+R}^{\prime}\\
\check{S}_{+R}^{\prime} \end{pmatrix}
-k\hat{D}_+(c_{2})
\begin{pmatrix} \check{\hat{N}}_{+L}^{\prime}\\ \check{N}_{+L}^{\prime}\\
\check{S}_{+L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 0 \cr 0 \cr 2{\mu}_{\bf 11}^{\prime}\check{S}_{-L}^{\prime}\end{pmatrix},\cr
\begin{matrix} {\rm (k)} \cr {\rm (l)} \cr {\rm (m)} \end{matrix} :\ &
k\hat{D}_+ (c_{2})
\begin{pmatrix} \check{\hat{N}}_{-R}^{\prime}\\ \check{N}_{-R}^{\prime}\\
\check{S}_{-R}^{\prime} \end{pmatrix}
+\sigma^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{-L}^{\prime}\\ \check{N}_{-L}^{\prime}\\
\check{S}_{-L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix} 0\\ 0\\
-\sqrt{2}\widetilde{\mu}_{2} \check{\nu}_{+R}^{\prime}
+2\mu_{\bf 11}^{\prime}\check{S}_{+R}^{\prime} \end{pmatrix},\cr
\begin{matrix} {\rm (n)} \cr {\rm (o)} \cr {\rm (p)} \end{matrix} :\ &
\overline{\sigma}^\mu\partial_\mu
\begin{pmatrix} \check{\hat{N}}_{-R}^{\prime}\\ \check{N}_{-R}^{\prime}\\
\check{S}_{-R}^{\prime} \end{pmatrix}
+ k\hat{D}_- (c_{2})
\begin{pmatrix} \check{\hat{N}}_{-L}^{\prime}\\ \check{N}_{-L}^{\prime}\\
\check{S}_{-L}^{\prime} \end{pmatrix}
=\delta(y)
\begin{pmatrix}
2\mu_{\bf 11}^{\prime}\check{\hat{N}}_{+L}^{\prime}\\
-2i\widetilde{\mu}_{2} \check{\nu}_{+L}
+2\mu_{\bf 11}^{\prime}\check{N}_{+L}^{\prime}\\
0 \end{pmatrix} \!, \cr
{\rm (q)}
:\ &
\Big\{ {\sigma}^\mu\partial_\mu\eta_-
- \frac{m_B}{\sqrt{k}} \, \nu_{+R}'-M\eta_-^C \Big\}
\delta(y)=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>Here, <inline-formula><tex-math notation="LaTeX" id="ImEquation762"><![CDATA[$\hat D_\pm (c)$]]></tex-math></inline-formula> is given by Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-2">4.2</xref>). To find the form of <inline-formula><tex-math notation="LaTeX" id="ImEquation763"><![CDATA[$\hat D_\pm (c_2)$]]></tex-math></inline-formula> for <inline-formula><tex-math notation="LaTeX" id="ImEquation764"><![CDATA[$(\hat N', N', S')$]]></tex-math></inline-formula> in <inline-formula><tex-math notation="LaTeX" id="ImEquation765"><![CDATA[$\Psi_{\bf 11}'$]]></tex-math></inline-formula>, let us recall that the original and twisted gauges are related by <inline-formula><tex-math notation="LaTeX" id="ImEquation766"><![CDATA[$\Psi_{\bf 11}' = \Omega (z) \widetilde \Psi_{\bf 11}' $]]></tex-math></inline-formula> where <inline-formula><tex-math notation="LaTeX" id="ImEquation767"><![CDATA[$\Omega (z) = e^{i\theta(z)T_{4,11}}$]]></tex-math></inline-formula>. Hence,
<disp-formula id="ptx175-M4-45"><label>(4.45)</label><tex-math notation="LaTeX" id="Equation108"><![CDATA[
\begin{align}
\psi_3' ~ &= \widetilde \psi_3' ,
\begin{pmatrix} \psi_4' \cr \psi_{11}' \end{pmatrix} =
\begin{pmatrix} \cos\theta(z) &\sin\theta(z) \cr -\sin\theta(z) &\cos\theta(z) \end{pmatrix}
\begin{pmatrix} \widetilde \psi_4' \cr \widetilde \psi_{11}' \end{pmatrix} .
\end{align}]]></tex-math></disp-formula></p>
<p>As
<disp-formula id="ptx175-M4-46"><label>(4.46)</label><tex-math notation="LaTeX" id="Equation109"><![CDATA[
\begin{align}
&\psi_{4}^{\prime}=\frac{1}{\sqrt{2}}
(\hat{N}^{\prime}-N^{\prime} ),\ \ \
\psi_{3}^{\prime}=\frac{i}{\sqrt{2}}
(\hat{N}^{\prime}+N^{\prime} ),\ \ \
\psi_{11}^{\prime}=S^{\prime} ,
\end{align}]]></tex-math></disp-formula>
one finds
<disp-formula id="ptx175-M4-47"><label>(4.47)</label><tex-math notation="LaTeX" id="Equation110"><![CDATA[
\begin{align}
&\begin{pmatrix} \widetilde{\check{\hat{N}}}{}' \cr \widetilde{\check N}' \cr \widetilde{\check S}' \end{pmatrix}
= \bar{\Omega} (z)
\begin{pmatrix}\check{ \hat{N}}{}' \cr \check N' \cr \check S' \end{pmatrix} \!, \cr
&\bar{\Omega}^{-1} (z) =
\begin{pmatrix}
\displaystyle \frac{1+\cos\theta(z)}{2}&\displaystyle \frac{1-\cos\theta(z)}{2}&
\displaystyle \frac{\sin\theta(z)}{\sqrt{2}} \cr
\displaystyle \frac{1-\cos\theta(z)}{2}&\displaystyle \frac{1+\cos\theta(z)}{2}&
-\displaystyle \frac{\sin\theta(z)}{\sqrt{2}} \cr
-\displaystyle \frac{\sin\theta(z)}{\sqrt{2}}&\displaystyle \frac{\sin\theta(z)}{\sqrt{2}}& \cos\theta(z)
\end{pmatrix} .
\end{align}]]></tex-math></disp-formula></p>
<p>It follows that
<disp-formula id="ptx175-M4-48"><label>(4.48)</label><tex-math notation="LaTeX" id="Equation111"><![CDATA[
\begin{align}
\hat D_- (c_2)
\begin{pmatrix} \check{\hat{N}}_{+R}' \cr \check N_{+R}' \cr \check S_{+R}' \end{pmatrix}
= \bar{\Omega}^{-1} (z) \, D_- (c_2)
\begin{pmatrix} \widetilde{\check{\hat{N}}}{}_{+R}' \cr \widetilde{\check N}{}_{+R}' \cr
\widetilde{\check S}{}_{+R}' \end{pmatrix} \!,
\end{align}]]></tex-math></disp-formula>
and so on.</p>
<p>To obtain boundary conditions at <inline-formula><tex-math notation="LaTeX" id="ImEquation768"><![CDATA[$y=0$]]></tex-math></inline-formula>, we integrate the above (a), (d), (e), (f), (j), (m), (n), and (o) (<inline-formula><tex-math notation="LaTeX" id="ImEquation769"><![CDATA[$\frac{1}{2}\int_{-\epsilon}^{+\epsilon}dy\cdots$]]></tex-math></inline-formula>) in the vicinity of <inline-formula><tex-math notation="LaTeX" id="ImEquation770"><![CDATA[$y=0$]]></tex-math></inline-formula> for parity-odd (in <inline-formula><tex-math notation="LaTeX" id="ImEquation771"><![CDATA[$x^5=y$]]></tex-math></inline-formula>) fields <inline-formula><tex-math notation="LaTeX" id="ImEquation772"><![CDATA[$\nu_{+R},\nu_{+L}',$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation773"><![CDATA[$\hat{N}_{+R}^{\prime},N_{+R}^{\prime},$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation774"><![CDATA[$S_{+L}^{\prime},$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation775"><![CDATA[$S_{-R}^{\prime},$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation776"><![CDATA[$\hat{N}_{-L}^{\prime},N_{-L}^{\prime}$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-M4-49"><label>(4.49)</label><tex-math notation="LaTeX" id="Equation112"><![CDATA[
\begin{align}
{\rm (a)}\ \Rightarrow\ &
+\check{\nu}_{+R}(x,\epsilon)
=i\widetilde{\mu}_2\check{N}_{-R}^{\prime}(x,0),\cr
{\rm (d)}\ \Rightarrow\ &
-\check{\nu}_{+L}^{\prime}(x,\epsilon)
=-\frac{\widetilde{\mu}_2}{\sqrt{2}}\check{S}_{-L}^{\prime}(x,0)
+ \frac{m_B}{2 \sqrt{k}} \, \eta_-(x),\cr
{\rm (e)}\ \Rightarrow\ &
+\check{\hat{N}}_{+R}^{\prime}(x,\epsilon)
=\mu_{\bf 11}^{\prime}\check{\hat{N}}_{-R}^{\prime}(x,0),\cr
{\rm (f)}\ \Rightarrow\ &
+\check{N}_{+R}^{\prime}(x,\epsilon)
=\mu_{\bf 11}^{\prime}\check{N}_{-R}^{\prime}(x,0),\cr
{\rm (j)}\ \Rightarrow\ &
-\check{S}_{+L}^{\prime}(x,\epsilon)
=\mu_{\bf 11}^{\prime}\check{S}_{-L}^{\prime}(x,0),\cr
{\rm (m)}\ \Rightarrow\ &
+\check{S}_{-R}^{\prime}(x,\epsilon)
=-\frac{\widetilde{\mu}_{2}}{\sqrt{2}}\check{\nu}_{+R}^{\prime}(x,0)
+\mu_{\bf 11}^{\prime}\check{S}_{+R}^{\prime}(x,0),\cr
{\rm (n)}\ \Rightarrow\ &
-\check{\hat{N}}_{-L}^{\prime}(x,\epsilon)
=\mu_{\bf 11}^{\prime}\check{\hat{N}}_{+L}^{\prime}(x,0),\cr
{\rm (o)}\ \Rightarrow\ &
-\check{N}_{-L}^{\prime}(x,\epsilon)
=-i\widetilde{\mu}_{2} \check{\nu}_{+L}(x,0)
+\mu_{\bf 11}^{\prime}\check{N}_{+L}^{\prime}(x,0).
\end{align}]]></tex-math></disp-formula></p>
<p>For parity-even (in <inline-formula><tex-math notation="LaTeX" id="ImEquation777"><![CDATA[$x^5=y$]]></tex-math></inline-formula>) fields <inline-formula><tex-math notation="LaTeX" id="ImEquation778"><![CDATA[$\nu_{+L},\nu_{+R}',$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation779"><![CDATA[$\hat{N}_{+L}^{\prime},N_{+L}^{\prime},$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation780"><![CDATA[$S_{+R}^{\prime},$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation781"><![CDATA[$S_{-L}^{\prime},$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation782"><![CDATA[$\hat{N}_{-R}^{\prime},N_{-R}^{\prime}$]]></tex-math></inline-formula>, we evaluate the equations of motion at <inline-formula><tex-math notation="LaTeX" id="ImEquation783"><![CDATA[$y=+\epsilon$]]></tex-math></inline-formula> by making use of the above conditions to find:
<disp-formula id="ptx175-M4-50"><label>(4.50)</label><tex-math notation="LaTeX" id="Equation113"><![CDATA[
\begin{align}
{\rm (b)}\ \Rightarrow\ &
-\hat{D}_-(c_{0})\check{\nu}_{+R}^{\prime}
-\frac{\widetilde{\mu}_{2}}{\sqrt{2}}\hat{D}_{+}\check{S}_{-R}^{\prime}
- \frac{m_B^2}{2k^2} \,\check{\nu}_{+R}^{\prime}
- \frac{m_B M}{2k^{3/2}} \, \eta_-^C
=0,\cr
{\rm (c)}\ \Rightarrow\ &
\hat{D}_+(c_{0})\check{\nu}_{+L}
+i\widetilde{\mu}_{2}\hat{D}_{-}\check{N}_{-L}^{\prime}
=0,\cr
{\rm (g)}\ \Rightarrow\ &
-\hat{D}_-(c_{0})\check{S}_{+R}^{\prime}
+\mu_{\bf 11}^{\prime}\hat{D}_{+}\check{S}_{-R}^{\prime}
=0,\cr
{\rm (h)}\ \Rightarrow\ &
\hat{D}_+(c_{0})\check{\hat{N}}_{+L}^{\prime}
+\mu_{\bf 11}^{\prime}\hat{D}_{-}\check{\hat{N}}_{-R}^{\prime}
=0,\cr
{\rm (i)}\ \Rightarrow\ &
\hat{D}_+(c_{0})\check{N}_{+L}^{\prime}
+\mu_{\bf 11}^{\prime}\hat{D}_{-}\check{N}_{-R}^{\prime}
=0,\cr
{\rm (k)}\ \Rightarrow\ &
\hat{D}_+(c_{2})\check{\hat{N}}_{-R}^{\prime}
-\mu_{\bf 11}^{\prime}\hat{D}_{-}\check{\hat{N}}_{+R}^{\prime}
=0,\cr
{\rm (l)}\ \Rightarrow\ &
\hat{D}_+(c_{2})\check{N}_{-R}^{\prime}
+i\widetilde{\mu}_{2} \hat{D}_{-}\check{\nu}_{+R}
-\mu_{\bf 11}^{\prime}\hat{D}_{-}\check{N}_{+R}^{\prime}
=0,\cr
{\rm (p)}\ \Rightarrow\ &
\hat{D}_-(c_{2})\check{S}_{-L}^{\prime}
-\frac{\widetilde{\mu}_{2}}{\sqrt{2}}\hat{D}_{+}\check{\nu}_{+L}'
+\mu_{\bf 11}^{\prime}\hat{D}_{+}\check{S}_{+L}^{\prime}
=0
\end{align}]]></tex-math></disp-formula>
at <inline-formula><tex-math notation="LaTeX" id="ImEquation784"><![CDATA[$y=+\epsilon$]]></tex-math></inline-formula>.</p>
<p>There are nine fields in the neutral fermion sector which intertwine with each other. To simplify the discussions, we consider the case in which <inline-formula><tex-math notation="LaTeX" id="ImEquation785"><![CDATA[$m_B^2/k^2, m_B M/k^2,\mu_{\bf 11}^{\prime}\gg\widetilde{\mu}_2$]]></tex-math></inline-formula>. In the <inline-formula><tex-math notation="LaTeX" id="ImEquation786"><![CDATA[$\widetilde{\mu}_2= 0$]]></tex-math></inline-formula> limit, the equations of motion and boundary conditions in the neutral sector, Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-44">4.44</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-49">4.49</xref>), and (<xref ref-type="disp-formula" rid="ptx175-M4-50">4.50</xref>), split into two parts. The first sector, the neutrino sector-1, contains <inline-formula><tex-math notation="LaTeX" id="ImEquation787"><![CDATA[$\check \nu_{+R}'$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation788"><![CDATA[$\check \nu_{+L}'$]]></tex-math></inline-formula> in <inline-formula><tex-math notation="LaTeX" id="ImEquation789"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation790"><![CDATA[$\eta_-$]]></tex-math></inline-formula> in <inline-formula><tex-math notation="LaTeX" id="ImEquation791"><![CDATA[$\chi_{\bf 1}$]]></tex-math></inline-formula>, while the second sector, the neutrino sector-2, contains other components in <inline-formula><tex-math notation="LaTeX" id="ImEquation792"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation793"><![CDATA[$\Psi_{\bf 11}^{\prime}$]]></tex-math></inline-formula>. <inline-formula><tex-math notation="LaTeX" id="ImEquation794"><![CDATA[$\widetilde{\mu}_2 \not= 0$]]></tex-math></inline-formula> mixes the neutrino sector-1 and sector-2.</p>
<p>In the twisted gauge, the mode functions are determined by the BCs on the IR brane. The mass spectra can be fixed by the BCs on the UV brane. For the neutrino sector-1, by using the boundary conditions at <inline-formula><tex-math notation="LaTeX" id="ImEquation795"><![CDATA[$z=z_L$]]></tex-math></inline-formula>, their mode functions can be written as
<disp-formula id="ptx175-M4-51"><label>(4.51)</label><tex-math notation="LaTeX" id="Equation114"><![CDATA[
\begin{align}
&\begin{pmatrix} \widetilde{\check{\nu}}_R \cr \widetilde{\check{\nu}}_R^{\prime} \cr
\eta_-^C \end{pmatrix}
= \begin{pmatrix} \alpha_\nu S_R(z;\lambda,c_0 ) \cr
i \alpha_{\nu'} C_R(z;\lambda,c_0 ) \cr - i \alpha_\eta^* / \sqrt{k} \end{pmatrix}
f_R(x) ,
\overline{\sigma}^\mu\partial_\mu f_R(x)=k\lambda f_L(x), \cr
&\begin{pmatrix} \widetilde{\check{\nu}}_L \cr \widetilde{\check{\nu}}_L^{\prime} \cr
\eta_- \end{pmatrix}
=\begin{pmatrix}
\alpha_\nu C_L(z;\lambda,c_0) \cr
i \alpha_{\nu'} S_L(z;\lambda,c_0) \cr i \alpha_\eta / \sqrt{k} \end{pmatrix}
f_L(x) ,
{\sigma}^\mu\partial_\mu f_L(x)=k\lambda f_R(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>Here, <inline-formula><tex-math notation="LaTeX" id="ImEquation796"><![CDATA[$f_{R/L}(x)$]]></tex-math></inline-formula> are chosen in the neutrino sector-1 such that <inline-formula><tex-math notation="LaTeX" id="ImEquation797"><![CDATA[$f_L^C(x)=-e^{i\delta_C}\sigma^2f_L(x)^*=f_R(x)$]]></tex-math></inline-formula> is satisfied where <inline-formula><tex-math notation="LaTeX" id="ImEquation798"><![CDATA[$\delta_C$]]></tex-math></inline-formula> is defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-11">2.11</xref>). One can take <inline-formula><tex-math notation="LaTeX" id="ImEquation799"><![CDATA[$\alpha_\nu , \alpha_{\nu'}, \alpha_\eta$]]></tex-math></inline-formula> to be real. In this case, <inline-formula><tex-math notation="LaTeX" id="ImEquation800"><![CDATA[$\sigma^\mu \partial_\mu \eta_- = -k\lambda \eta_-^C$]]></tex-math></inline-formula> is satisfied so that equation (q) in Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-44">4.44</xref>) implies that
<disp-formula id="ptx175-M4-52"><label>(4.52)</label><tex-math notation="LaTeX" id="Equation115"><![CDATA[
\begin{align}
\frac{m_B}{\sqrt{k}} \, \check \nu_{+R}' \big|_{y=0} + (k\lambda + M) \eta_-^C =0 .
\end{align}]]></tex-math></disp-formula></p>
<p>With this identity the first relation in Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-50">4.50</xref>) can be rewritten as
<disp-formula id="ptx175-M4-53"><label>(4.53)</label><tex-math notation="LaTeX" id="Equation116"><![CDATA[
\begin{align}
&
\hat{D}_-(c_{0})\check{\nu}_{+R}^{\prime}
+\frac{\widetilde{\mu}_{2}}{\sqrt{2}}\hat{D}_{+}\check{S}_{-R}^{\prime}
- \frac{m_B \lambda }{ 2\sqrt{k} } \, \eta_-^C =0.
\end{align}]]></tex-math></disp-formula></p>
<p>Setting <inline-formula><tex-math notation="LaTeX" id="ImEquation801"><![CDATA[$\widetilde{\mu}_{2} =0$]]></tex-math></inline-formula>, one finds that the BCs at <inline-formula><tex-math notation="LaTeX" id="ImEquation802"><![CDATA[$z=1^+$]]></tex-math></inline-formula> in the twisted gauge can be written as
<disp-formula id="ptx175-M4-54"><label>(4.54)</label><tex-math notation="LaTeX" id="Equation117"><![CDATA[
\begin{align}
K_{\nu_1}
\begin{pmatrix} \alpha_\nu \cr
\alpha_{\nu'} \cr \alpha_\eta \end{pmatrix}
=
\begin{pmatrix}
\cos \displaystyle \frac{\theta_H}{2}S_R^0 &\sin \displaystyle \frac{\theta_H}{2}C_R^0 &0 \cr
-\sin \displaystyle \frac{\theta_H}{2}C_L^0 &\cos \displaystyle \frac{\theta_H}{2}S_L^0
&\displaystyle \frac{m_B}{2k} \cr
-\sin \displaystyle \frac{\theta_H}{2}S_R^0 &\cos\displaystyle \frac{\theta_H}{2}C_R^0
& - \displaystyle \frac{k\lambda + M}{m_B} \end{pmatrix}
\begin{pmatrix} \alpha_\nu \cr
\alpha_{\nu'} \cr \alpha_\eta \end{pmatrix} = 0 .
\end{align}]]></tex-math></disp-formula></p>
<p>From <inline-formula><tex-math notation="LaTeX" id="ImEquation803"><![CDATA[$\mbox{det}\,K_{\nu_1}=0$]]></tex-math></inline-formula>, we find the mass spectrum formula for the neutrino sector-1:
<disp-formula id="ptx175-M4-55"><label>(4.55)</label><tex-math notation="LaTeX" id="Equation118"><![CDATA[
\begin{align}
\det K_{\nu_1} = - \frac{k\lambda+M}{m_B} \left\{S_L^0 S_R^0+\sin^2\frac{\theta_H}{2}\right\}
- \frac{m_B}{2k } \, S_R^0 C_R^0=0.
\end{align}]]></tex-math></disp-formula></p>
<p>This gives the gauge&#x2013;Higgs seesaw mechanism for the small neutrino masses. For <inline-formula><tex-math notation="LaTeX" id="ImEquation804"><![CDATA[$\lambda z_L\ll 1$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation805"><![CDATA[$k\lambda\ll |M|$]]></tex-math></inline-formula>, the neutrino mass is given by
<disp-formula id="ptx175-M4-56"><label>(4.56)</label><tex-math notation="LaTeX" id="Equation119"><![CDATA[
\begin{align}
m_\nu \simeq -\frac{2m_u^2 Mz_L^{2c_0+1}}{(2c_0+1)m_B^{2}},
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation806"><![CDATA[$m_u$]]></tex-math></inline-formula> is given by Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-23">4.23</xref>). As shown in Ref. [<xref ref-type="bibr" rid="B38">38</xref>], the gauge&#x2013;Higgs seesaw mechanism is characterized by a <inline-formula><tex-math notation="LaTeX" id="ImEquation807"><![CDATA[$3 \times 3$]]></tex-math></inline-formula> mass matrix
<disp-formula id="ptx175-M4-57"><label>(4.57)</label><tex-math notation="LaTeX" id="Equation120"><![CDATA[
\begin{align}
\begin{pmatrix} & m_D &\cr m_D && \tilde m_B \cr & \tilde m_B & M \end{pmatrix}
\end{align}]]></tex-math></disp-formula>
in the 4D effective theory in which <inline-formula><tex-math notation="LaTeX" id="ImEquation808"><![CDATA[$m_D = m_u$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation809"><![CDATA[$\tilde m_B \sim m_B z_L^{- c_0 -1/2}$]]></tex-math></inline-formula>. The Majorana mass <inline-formula><tex-math notation="LaTeX" id="ImEquation810"><![CDATA[$M$]]></tex-math></inline-formula> may take a moderate value.</p>
<p>Next, we consider the neutrino sector-2. The BCs at the IR brane determine the mode functions in the twisted gauge to be
<disp-formula id="ptx175-M4-58"><label>(4.58)</label><tex-math notation="LaTeX" id="Equation121"><![CDATA[
\begin{align}
&\begin{pmatrix}
\tilde{\check{\hat{N}}}_{+R}^{\prime}\\
\tilde{\check{N}}_{+R}^{\prime}\\
\tilde{\check{S}}_{+R}^{\prime}\\
\tilde{\check{\hat{N}}}_{-R}^{\prime}\\
\tilde{\check{N}}_{-R}^{\prime}\\
\tilde{\check{S}}_{-R}^{\prime}\\
\end{pmatrix}
=\begin{pmatrix}
\alpha_{\hat{N}_+^{\prime}}S_R(z;\lambda,c_2)\\
\alpha_{N_+^{\prime}}S_R(z;\lambda,c_2)\\
\alpha_{S_+^{\prime}}C_R(z;\lambda,c_2)\\
\alpha_{\hat{N}_-^{\prime}}C_L(z;\lambda,c_2)\\
\alpha_{N_-^{\prime}}C_L(z;\lambda,c_2)\\
\alpha_{S_-^{\prime}}S_L(z;\lambda,c_2)\\
\end{pmatrix} f_R(x) , \cr
&\begin{pmatrix}
\tilde{\check{\hat{N}}}_{+L}^{\prime}\\
\tilde{\check{N}}_{+L}^{\prime}\\
\tilde{\check{S}}_{+L}^{\prime}\\
\tilde{\check{\hat{N}}}_{-L}^{\prime}\\
\tilde{\check{N}}_{-L}^{\prime}\\
\tilde{\check{S}}_{-L}^{\prime}\\
\end{pmatrix}
= \begin{pmatrix}
\alpha_{\hat{N}_+^{\prime}}C_L(z;\lambda,c_2)\\
\alpha_{N_+^{\prime}}C_L(z;\lambda,c_2)\\
\alpha_{S_+^{\prime}}S_L(z;\lambda,c_2)\\
-\alpha_{\hat{N}_-^{\prime}}S_R(z;\lambda,c_2)\\
-\alpha_{N_-^{\prime}}S_R(z;\lambda,c_2)\\
-\alpha_{S_-^{\prime}}C_R(z;\lambda,c_2)\\
\end{pmatrix} f_L(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>By making use of Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-47">4.47</xref>) and (<xref ref-type="disp-formula" rid="ptx175-M4-48">4.48</xref>), the BCs at the UV brane are expressed as
<disp-formula id="ptx175-M4-59"><label>(4.59)</label><tex-math notation="LaTeX" id="Equation122"><![CDATA[
\begin{align}
K_{\nu_2}
\left(
\alpha_{\hat{N}_+^{\prime}}\
\alpha_{N_+^{\prime}}\
\alpha_{S_+^{\prime}}\
\alpha_{\hat{N}_-^{\prime}}\
\alpha_{N_-^{\prime}}\
\alpha_{S_-^{\prime}}\
\right)^{\rm T}
=0 ,
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M4-60"><label>(4.60)</label><tex-math notation="LaTeX" id="Equation123"><![CDATA[
\begin{align}
K_{\nu_2}&= \begin{pmatrix} A & -\mu_{\bf 11}' B \cr
-\mu_{\bf 11}' B & A \end{pmatrix} \!, \cr
A &= \begin{pmatrix}
\displaystyle \frac{1+\cos\theta_H}{2}S_R^2
&\displaystyle \frac{1-\cos\theta_H}{2}S_R^2
&\displaystyle \frac{\sin\theta_H}{\sqrt{2}} C_R^2 \cr
\displaystyle \frac{1-\cos\theta_H}{2}S_R^2
&\displaystyle \frac{1+\cos\theta_H}{2}S_R^2
&-\displaystyle \frac{\sin\theta_H}{\sqrt{2}} C_R^2 \cr
-\displaystyle \frac{\sin\theta_H}{\sqrt{2}}C_L^2
&\displaystyle \frac{\sin\theta_H}{\sqrt{2}}C_L^2
&\cos\theta_H S_L^2 \end{pmatrix} \!, \cr
B &= \begin{pmatrix}
\displaystyle \frac{1+\cos\theta_H}{2}C_L^2
&\displaystyle \frac{1-\cos\theta_H}{2}C_L^2
&\displaystyle \frac{\sin\theta_H}{\sqrt{2}} S_L^2 \cr
\displaystyle \frac{1-\cos\theta_H}{2}C_L^2
&\displaystyle \frac{1+\cos\theta_H}{2}C_L^2
&-\displaystyle \frac{\sin\theta_H}{\sqrt{2}} S_L^2 \cr
-\displaystyle \frac{\sin\theta_H}{\sqrt{2}}S_R^2
&\displaystyle \frac{\sin\theta_H}{\sqrt{2}}S_R^2
&\cos\theta_H C_R^2 \end{pmatrix} .
\end{align}]]></tex-math></disp-formula></p>
<p>Note that
<disp-formula id="ptx175-M4-61"><label>(4.61)</label><tex-math notation="LaTeX" id="Equation124"><![CDATA[
\begin{align}
&K_{\nu_2}' = \begin{pmatrix} V & 0 \cr 0 &V \end{pmatrix} K_{\nu_2}
\begin{pmatrix} V^{-1} & 0 \cr 0 &V^{-1} \end{pmatrix}
= \begin{pmatrix} A' & -\mu_{\bf 11}' B' \cr
-\mu_{\bf 11}' B' & A' \end{pmatrix} \!, \cr
&V = V^{-1} = \begin{pmatrix} 2^{-1/2} & 2^{-1/2} & 0 \cr
2^{-1/2} & - 2^{-1/2} & 0 \cr 0 &0 &1 \end{pmatrix} \!, \cr
&A' = \begin{pmatrix} S_R^2 & 0 &0 \cr
0 & \cos \theta_H S_R^2 & \sin \theta_H C_R^2 \cr
0 & - \sin\theta_H C_L^2 & \cos\theta_H S_L^2 \end{pmatrix} \!, \cr
&B' = \begin{pmatrix} C_L^2 & 0 &0 \cr
0 & \cos \theta_H C_L^2 & \sin \theta_H S_L^2 \cr
0 & - \sin\theta_H S_R^2 & \cos\theta_H C_R^2 \end{pmatrix} .
\end{align}]]></tex-math></disp-formula></p>
<p>From <inline-formula><tex-math notation="LaTeX" id="ImEquation811"><![CDATA[$\mbox{det}\,K_{\nu_2}= \mbox{det}\,K_{\nu_2}'= 0$]]></tex-math></inline-formula> the mass spectra for the neutrino sector-2 are found to be
<disp-formula id="ptx175-M4-62"><label>(4.62)</label><tex-math notation="LaTeX" id="Equation125"><![CDATA[
\begin{align}
&\det K_{\nu_2} =\left(S_R^2 S_R^2-\mu_{\bf 11}^{\prime 2}C_L^2C_L^2\right)
\Big\{
\left(S_L^2S_R^2 +\sin^2\theta_H\right)^2 \cr
&\hskip 2.cm
+\mu_{\bf 11}^{\prime 2}
\left(2\sin^2\theta_H\cos^2\theta_H
-C_R^2C_R^2S_R^2S_R^2
-C_L^2C_L^2S_L^2S_L^2\right) \cr
&\hskip 2.cm
+\mu_{\bf 11}^{\prime 4}
\left(S_L^2 S_R^2+\cos^2\theta_H\right)^2
\Big\} = 0 .
\end{align}]]></tex-math></disp-formula></p>
<p>In the <inline-formula><tex-math notation="LaTeX" id="ImEquation812"><![CDATA[$\mu_{\bf 11}^{\prime}=0$]]></tex-math></inline-formula> limit, or in the absence of the brane interactions, Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-62">4.62</xref>) becomes
<disp-formula id="ptx175-M4-63"><label>(4.63)</label><tex-math notation="LaTeX" id="Equation126"><![CDATA[
\begin{align}
\left(S_R^2\right)^2\left(S_L^2 S_R^2+\sin^2\theta_H\right)^2=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>For large <inline-formula><tex-math notation="LaTeX" id="ImEquation813"><![CDATA[$\mu_{\bf 11}^{\prime}$]]></tex-math></inline-formula>, it becomes
<disp-formula id="ptx175-M4-64"><label>(4.64)</label><tex-math notation="LaTeX" id="Equation127"><![CDATA[
\begin{align}
\left(C_L^2\right)^2\left(S_L^2 S_R^2+\cos^2\theta_H\right)^2
\simeq 0 .
\end{align}]]></tex-math></disp-formula></p>
<p>For <inline-formula><tex-math notation="LaTeX" id="ImEquation814"><![CDATA[${\widetilde{\mu}}_2 \not= 0$]]></tex-math></inline-formula> the neutrino sector-1 and sector-2 mix through the boundary conditions. One needs to solve
<disp-formula id="ptx175-M4-65"><label>(4.65)</label><tex-math notation="LaTeX" id="Equation128"><![CDATA[
\begin{align}
&K \,
\big( \alpha_\nu \ \alpha_{\nu'} \ \alpha_\eta \
\alpha_{\hat{N}_+^{\prime}}\ \alpha_{N_+^{\prime}}\ \alpha_{S_+^{\prime}}\
\alpha_{\hat{N}_-^{\prime}}\ \alpha_{N_-^{\prime}}\ \alpha_{S_-^{\prime}} \big)^T
=0 , \cr
&K= \begin{pmatrix} K_{\nu_1} & 0 & -i {\widetilde \mu}_2\, D \cr
0 &A & -\mu_{\bf 11}' B \cr
i {\widetilde \mu}_2 \, C & -\mu_{\bf 11}' B & A \end{pmatrix} , \cr
&C = \begin{pmatrix} 0 & 0 & 0 \cr
\cos \frac{1}{2} \theta_H C_L^0 & \sin \frac{1}{2} \theta_H S_L^0 & 0 \cr
- \displaystyle \frac{\sin \frac{1}{2} \theta_H}{\sqrt{2}} S_R^0
& \displaystyle \frac{\cos \frac{1}{2} \theta_H}{\sqrt{2}} C_R^0 & 0 \end{pmatrix} , \cr
&D = \begin{pmatrix}
\displaystyle \frac{1-\cos\theta_H}{2}C_L^2
&\displaystyle \frac{1+\cos\theta_H}{2}C_L^2
&-\displaystyle \frac{\sin\theta_H}{\sqrt{2}} S_L^2 \cr
-\displaystyle \frac{\sin\theta_H}{2}S_R^2
&\displaystyle \frac{\sin\theta_H}{2}S_R^2
&\displaystyle \frac{\cos\theta_H}{\sqrt{2}} C_R^2 \cr
0 & 0 & 0 \end{pmatrix} .
\end{align}]]></tex-math></disp-formula>
<inline-formula><tex-math notation="LaTeX" id="ImEquation815"><![CDATA[$\det K$]]></tex-math></inline-formula> is evaluated to be
<disp-formula id="ptx175-M4-66"><label>(4.66)</label><tex-math notation="LaTeX" id="Equation129"><![CDATA[
\begin{align}
&\det K = F_0 + F_2 \, ({\widetilde \mu}_2)^2 + F_4\, ({\widetilde \mu}_2)^4 , \cr
&F_0 = \det K_{\nu_1} \cdot \det K_{\nu_2} , \cr
&F_2 = \bigg\{ \frac{m_B}{4k} (S_L^0 S_R^0 + \cos^2 \frac{1}{2} \theta_H )
+ \frac{k\lambda +M}{2 m_B} C_L^0 S_L^0 \bigg\} \cr
&\qquad
\times \bigg\{ C_L^2 S_R^2 (2 X + \sin^2 \theta_H ) (X + \sin^2 \theta_H ) \cr
&\qquad
- \mu_{11}^{\prime \, 2} \Big[ (C_L^2)^3 S_L^2 ( 2X + \sin^2 \theta_H )
+ C_R^2 (S_R^2)^3 (2X + 1 + \cos^2 \theta_H)\cr
&\qquad
+ C_L^2 S_R^2 ( X^2 - 2) \sin^2 \theta_H \cos^2 \theta_H \Big]
+ \mu_{11}^{\prime \, 4} C_L^2 S_R^2 (X + \cos^2 \theta_H) (2X + 1 + \cos^2\theta_H) \bigg\}
\cr
&\quad
+\frac{k\lambda + M}{2 m_B} \big\{ (S_R^2)^2 - \mu_{11}^{\prime \, 2} (C_L^2)^2 \big\}
\bigg\{ \Big[ (1 + \mu_{11}^{\prime \, 2} ) X + \sin^2 \theta_H
+ \mu_{11}^{\prime \, 2} \cos^2 \theta_H \Big] \cos \theta_H \sin^2 \theta_H
\cr
&\qquad
+ C_R^0 S_R^0 \Big[ C_R^2 S_R^2 (X + \sin^2\theta_H ) -
\mu_{11}^{\prime \, 2} C_L^2 S_L^2 (X + \cos^2 \theta_H ) \Big] \bigg\},
\cr
&F_4 = - \frac{k \lambda + M}{32 m_B} (S_L^0 S_R^0 + \cos^2 \frac{1}{2} \theta_H ) \cr
&\quad
\times \Big[ (S_R^2 )^2 \big\{ 16 (S_L^2 S_R^2)^2 + (20 - 4 \cos 2\theta_H) S_L^2 S_R^2
+ 5 - 4 \cos 2\theta_H - \cos 4\theta_H \big\} \cr
&\qquad
- \mu_{11}^{\prime \, 2} (C_L^2)^2 \big\{ 16 (S_L^2 S_R^2)^2
+ (12 + 4 \cos 2\theta_H) S_L^2 S_R^2 - \cos 4\theta_H \big\} \Big],
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation816"><![CDATA[$X = S_L^2 S_R^2$]]></tex-math></inline-formula>. In evaluating the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation817"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula>, we shall use the approximate formula <inline-formula><tex-math notation="LaTeX" id="ImEquation818"><![CDATA[$\det K \sim \det K_{\nu_1} \det K_{\nu_2}$]]></tex-math></inline-formula> for small <inline-formula><tex-math notation="LaTeX" id="ImEquation819"><![CDATA[$\widetilde{\mu}_2^{2}$]]></tex-math></inline-formula>.</p>
</sec>
<sec id="SEC4.5"><title>4.5. Dark fermion</title>
<p>(v) <inline-formula><tex-math notation="LaTeX" id="ImEquation820"><![CDATA[$\Psi_{\bf 32}^{\prime} = \Psi_{\bf 32}^{\alpha = 4}$]]></tex-math></inline-formula></p>
<p>The 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation821"><![CDATA[$SO(11)$]]></tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX" id="ImEquation822"><![CDATA[${\bf 32}$]]></tex-math></inline-formula> Weyl fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation823"><![CDATA[$\Psi_{\bf 32}^{\prime}$]]></tex-math></inline-formula>, which may be called a dark fermion multiplet, has sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation824"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes. For <inline-formula><tex-math notation="LaTeX" id="ImEquation825"><![CDATA[$\Psi_{\bf 32}'$]]></tex-math></inline-formula> one need not introduce brane interactions. We consider the action in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-19">2.19</xref>) for the dark fermion sector <inline-formula><tex-math notation="LaTeX" id="ImEquation826"><![CDATA[$\Psi_{\bf 32}^{\prime}$]]></tex-math></inline-formula>. From the parity assignment shown in <xref ref-type="table" rid="T2">Table 2</xref>, the mass formula for the dark fermions is found to be
<disp-formula id="ptx175-M4-67"><label>(4.67)</label><tex-math notation="LaTeX" id="Equation130"><![CDATA[
\begin{align}
S_L^{0'}S_R^{0'}+\cos^2\frac{\theta_H}{2}=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>For <inline-formula><tex-math notation="LaTeX" id="ImEquation827"><![CDATA[$\lambda z_L\ll 1$]]></tex-math></inline-formula>, the dark fermion mass is given by
<disp-formula id="ptx175-M4-68"><label>(4.68)</label><tex-math notation="LaTeX" id="Equation131"><![CDATA[
\begin{align}
m_{\rm Dark}\simeq
\begin{cases}
k z_L^{-1}\sqrt{1-4c_{0}^{\prime 2}} \, \cos\displaystyle \frac{\theta_H}{2}&
\mbox{for}\ c_{0}'<1/2 , \cr
k z_L^{-1/2-c_{0}'}\sqrt{4c_{0}^{\prime 2}-1} \, \cos\displaystyle \frac{\theta_H}{2}&
\mbox{for}\ c_{0}' >1/2 .
\end{cases}
\end{align}]]></tex-math></disp-formula></p>
<p>As in the case of the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation828"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT discussed in Ref. [<xref ref-type="bibr" rid="B35">35</xref>], the bulk mass parameter of the dark fermions, <inline-formula><tex-math notation="LaTeX" id="ImEquation829"><![CDATA[$c_{0}'$]]></tex-math></inline-formula>, must be relatively small not to become light exotic particles.</p>
</sec>
</sec>
<sec id="SEC5"><title>5. Effective potential</title>
<p>We evaluate the Higgs effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation830"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> by using the mass spectrum formulas of the sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation831"><![CDATA[$n=0$]]></tex-math></inline-formula> KK modes of the <inline-formula><tex-math notation="LaTeX" id="ImEquation832"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge bosons and fermions. The evaluation is done in the same manner as in Ref. [<xref ref-type="bibr" rid="B35">35</xref>].</p>
<p>One-loop effective potential from each KK tower is given by [<xref ref-type="bibr" rid="B9">9</xref>,<xref ref-type="bibr" rid="B45">45</xref>,<xref ref-type="bibr" rid="B46">46</xref>]:
<disp-formula id="ptx175-M5-1"><label>(5.1)</label><tex-math notation="LaTeX" id="Equation132"><![CDATA[
\begin{align}
V_{\rm eff}(\theta_H)&=
\pm\frac{1}{2}\int\frac{d^4p}{(2\pi)^4}
\sum_n \mbox{ln}\left(p^2+m_n(\theta_H)^2\right) ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation833"><![CDATA[$m_n(\theta_H)$]]></tex-math></inline-formula> is the mass spectrum of the KK tower and we take the <inline-formula><tex-math notation="LaTeX" id="ImEquation834"><![CDATA[$+$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation835"><![CDATA[$-$]]></tex-math></inline-formula> sign for bosons and fermions, respectively. When the mass spectrum <inline-formula><tex-math notation="LaTeX" id="ImEquation836"><![CDATA[$\{ m_n = k \lambda_n \}$]]></tex-math></inline-formula> is determined by <inline-formula><tex-math notation="LaTeX" id="ImEquation837"><![CDATA[$1 + \tilde Q (\lambda_n) f(\theta_H) = 0$]]></tex-math></inline-formula>, Eq. (<xref ref-type="disp-formula" rid="ptx175-M5-1">5.1</xref>) can be written as
<disp-formula id="ptx175-M5-2"><label>(5.2)</label><tex-math notation="LaTeX" id="Equation133"><![CDATA[
\begin{align}
V_{\rm eff}(\theta_H) &= \pm I\left[Q(q);f(\theta_H)\right] \cr
I\left[Q(q);f(\theta_H)\right] &= \frac{(k z_L^{-1} )^4}{(4\pi)^2}
\int_0^\infty dq \, q^3 \ln [ 1 + Q(q) f (\theta_H) ] ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation838"><![CDATA[$Q(q) = \tilde Q (iq z_L^{-1} )$]]></tex-math></inline-formula>. We utilize the mass spectra <inline-formula><tex-math notation="LaTeX" id="ImEquation839"><![CDATA[$m_n(\theta_H)$]]></tex-math></inline-formula> of the <inline-formula><tex-math notation="LaTeX" id="ImEquation840"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk gauge and fermions obtained in <xref ref-type="sec" rid="SEC3">Sects. 3</xref> and <xref ref-type="sec" rid="SEC4">4</xref>. Note that only <inline-formula><tex-math notation="LaTeX" id="ImEquation841"><![CDATA[$\theta_H$]]></tex-math></inline-formula>-dependent mass spectra contribute to the <inline-formula><tex-math notation="LaTeX" id="ImEquation842"><![CDATA[$\theta_H$]]></tex-math></inline-formula>-dependent part of <inline-formula><tex-math notation="LaTeX" id="ImEquation843"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula>. It contains the <inline-formula><tex-math notation="LaTeX" id="ImEquation844"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk gauge field, <inline-formula><tex-math notation="LaTeX" id="ImEquation845"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor, and vector fermion fields.</p>
<p>We summarize those mass spectra. The <inline-formula><tex-math notation="LaTeX" id="ImEquation846"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge boson contribution is almost the same as in the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation847"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT. The difference from the 5D case is that the <inline-formula><tex-math notation="LaTeX" id="ImEquation848"><![CDATA[$Y$]]></tex-math></inline-formula> boson contributions can be neglected in the current scheme as they have masses of <inline-formula><tex-math notation="LaTeX" id="ImEquation849"><![CDATA[$O(m_{{\rm KK}_6})$]]></tex-math></inline-formula>. The relevant mass spectra of <inline-formula><tex-math notation="LaTeX" id="ImEquation850"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge fields <inline-formula><tex-math notation="LaTeX" id="ImEquation851"><![CDATA[$A_\mu$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation852"><![CDATA[$A_z$]]></tex-math></inline-formula> are given, from Eqs. (<xref ref-type="disp-formula" rid="ptx175-M3-7">3.7</xref>), (<xref ref-type="disp-formula" rid="ptx175-M3-12">3.12</xref>), and (<xref ref-type="disp-formula" rid="ptx175-M3-18">3.18</xref>), by
<disp-formula id="ptx175-M5-3"><label>(5.3)</label><tex-math notation="LaTeX" id="Equation134"><![CDATA[
\begin{align}
&W^{\pm}\,\mbox{tower}:\
1+\frac{\lambda}{2C'(1;\lambda)S(1;\lambda)}\sin^2\theta_H=0 , \cr
&Z\,\mbox{tower}:\
1+\frac{\lambda}{2 \cos^2 \theta_W \, C'(1;\lambda)S(1;\lambda)}
\sin^2\theta_H=0 , \cr
&A_z^{a4},A_z^{a\, 11}\ (a=1,2,3) :\
1+\frac{\lambda}{C'(1;\lambda)S(1;\lambda)}\sin^2\theta_H=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>The mass spectra of up- and down-type quarks, charged leptons, neutrinos, and dark fermions are given, from Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-22">4.22</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-30">4.30</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-39">4.39</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-55">4.55</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-62">4.62</xref>), and (<xref ref-type="disp-formula" rid="ptx175-M4-67">4.67</xref>), by
<disp-formula id="ptx175-M5-4"><label>(5.4)</label><tex-math notation="LaTeX" id="Equation135"><![CDATA[
\begin{align}
&\mbox{(i)}:\ Q_{\rm EM}=+\frac{2}{3}:\ \
1+\frac{\sin^2\frac{1}{2} \theta_H}{S_L^{0}S_R^{0}}=0 , \cr
&\mbox{(ii)}:\ Q_{\rm EM}=-\frac{1}{3}:\ \
1+\frac{\sin^2\frac{1}{2} \theta_H}{S_L^0 S_R^0
+\displaystyle \frac{\mu_1^2 S_R^0 C_R^0 S_L^1 C_R^1}
{\mu_{\bf 11}^2(C_R^1)^2-(S_L^1)^2}} = 0 , \cr
&\mbox{(iii)}:\ Q_{\rm EM}=-1:\ \
1+ \frac{\sin^2\frac{1}{2} \theta_H}{S_L^0 S_R^0
+\displaystyle \frac{\widetilde{\mu}_2^2 S_L^0 C_L^0 S_R^2 C_L^2}
{\mu_{\bf 11}^{\prime 2}(C_L^2)^2-(S_R^2)^2}} =0 , \cr
&\mbox{(iv-1)}:\ Q_{\rm EM}=0:\ \
1+\frac{\sin^2\frac{1}{2} \theta_H}{S_L^0 S_R^0
+\displaystyle \frac{m_B^2 /k }{2(k\lambda+M)}S_R^0 C_R^0} =0 , \cr
&\mbox{(iv-2)}:\ Q_{\rm EM}=0:\ \
1 + \frac{f_1 (\theta_H)}{f_2 } = 0 , \cr
&\hskip .5cm
f_1 (\theta_H) = (1 - \mu_{\bf 11}^{\prime \, 2} )
\big\{ 2 (1 + \mu_{\bf 11}^{\prime \, 2}) S_L^2S_R^2
+2\mu_{\bf 11}^{\prime \, 2}
+( 1 - \mu_{\bf 11}^{\prime \, 2}) \sin^2\theta_H \big\} \sin^2 \theta_H , \cr
&\hskip .5cm
f_2 = \mu_{\bf 11}^{\prime 4}
+2\mu_{\bf 11}^{\prime 4}S_L^2S_R^2
+(1+\mu_{\bf 11}^{\prime 4})(S_L^2S_R^2)^2
-\mu_{\bf 11}^{\prime 2}\left\{(C_R^2S_R^2)^2+(C_L^2S_L^2)^2\right\} , \cr
&
\mbox{(v)}:\ Q_{\rm EM}=+\frac{2}{3},-\frac{2}{3},-1,0:\ \
1+\frac{\cos^2 \frac{1}{2} \theta_H}{S_L^{0'}S_R^{0'}} =0 ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation853"><![CDATA[$m_B^2/k^2, m_B M/k^2, \mu_{\bf 11}' \gg \widetilde \mu_2$]]></tex-math></inline-formula> has been assumed for (iv-1) and (iv-2). (v) is the dark fermion mass spectrum. <inline-formula><tex-math notation="LaTeX" id="ImEquation854"><![CDATA[$c_0'$]]></tex-math></inline-formula> stands for the bulk mass of the 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation855"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor bulk Weyl fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation856"><![CDATA[$\Psi_{\bf 32}^{\alpha=4}$]]></tex-math></inline-formula>.</p>
<p>We evaluate the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation857"><![CDATA[$V_{\rm eff}(\theta_H) = V_{\rm eff}^{\rm gauge}(\theta_H) +V_{\rm eff}^{\rm fermion}(\theta_H)$]]></tex-math></inline-formula>. In the <inline-formula><tex-math notation="LaTeX" id="ImEquation858"><![CDATA[$R_\xi$]]></tex-math></inline-formula> gauge <inline-formula><tex-math notation="LaTeX" id="ImEquation859"><![CDATA[$V_{\rm eff}^{\rm gauge}(\theta_H)$]]></tex-math></inline-formula> is decomposed into
<disp-formula id="ptx175-M5-5"><label>(5.5)</label><tex-math notation="LaTeX" id="Equation136"><![CDATA[
\begin{align}
&V_{\rm eff}^{\rm gauge}(\theta_H)=
V_{\rm eff}^{W^{\pm}} +V_{\rm eff}^{Z}
+V_{\rm eff}^{A_z^{a4},A_z^{a,11}},\cr
&V_{\rm eff}^{W^{\pm}}(\theta_H)
=2(3-\xi^2)
I\left[\frac{1}{2} Q_{0} \left(q,1\right);\sin^2\theta_H\right] , \cr
&V_{\rm eff}^{Z}(\theta_H)
=(3-\xi^2)
I \bigg[ \frac{Q_{0}\left(q,1\right)}{2 \cos^2 \theta_W}; \sin^2\theta_H \bigg], \cr
&V_{\rm eff}^{A_z^{a4},A_z^{a\, 11}}(\theta_H)
=3\xi^2
I\left[Q_{0}\left(q,1\right);\sin^2\theta_H\right].
\end{align}]]></tex-math></disp-formula></p>
<p>Here,
<disp-formula id="ptx175-M5-6"><label>(5.6)</label><tex-math notation="LaTeX" id="Equation137"><![CDATA[
\begin{align}
&Q_0\left(q,c\right) =\frac{z_L}{q^2}
\frac{1}{\hat{F}_{c}^{++}(q)\hat{F}_{c}^{-}(q)} , \cr
&\hat{F}_{c}^{\pm \pm}(q)
= \hat F_{c\pm \frac{1}{2}, c\pm \frac{1}{2}} (q z_L^{-1}, q) , \cr
&\hat F_{\alpha, \beta} (u,v) = I_\alpha (u) K_\beta (v) -
e^{-i (\alpha -\beta) \pi} K_\alpha (u) I_\beta (v) ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation860"><![CDATA[$I_\alpha (u)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation861"><![CDATA[$K_\beta (u)$]]></tex-math></inline-formula> are modified Bessel functions.</p>
<p>The fermion part <inline-formula><tex-math notation="LaTeX" id="ImEquation862"><![CDATA[$V_{\rm eff}^{\rm fermion}(\theta_H)$]]></tex-math></inline-formula> is evaluated in a similar manner. Following the decomposition in Eq. (<xref ref-type="disp-formula" rid="ptx175-M5-4">5.4</xref>), and taking into account four degrees of freedom for each Dirac fermion and the color factor 3 for quarks, one can write:
<disp-formula id="ptx175-M5-7"><label>(5.7)</label><tex-math notation="LaTeX" id="Equation138"><![CDATA[
\begin{align}
V_{\rm eff}^{\rm fermion}(\theta_H)
& =V_{\rm eff}^{\rm (i)}+V_{\rm eff}^{\rm (ii)}
+V_{\rm eff}^{\rm (iii)}
+V_{\rm eff}^{\rm (iv-1)}
+V_{\rm eff}^{\rm (iv-2)}
+V_{\rm eff}^{\rm (v)}, \cr
V_{\rm eff}^{\rm (i)}(\theta_H)
&=-12I \left[Q_0\left(q,c_0\right);\sin^2\frac{1}{2} \theta_H \right] , \cr
V_{\rm eff}^{\rm (ii)}(\theta_H)
&=-12I\left[Q_{\rm (ii)}
(q,c_0,c_1,\mu_1,\mu_{\bf 11});\sin^2 \frac{1}{2} \theta_H \right] ,\cr
V_{\rm eff}^{\rm (iii)}(\theta_H)
&=-4I\left[Q_{\rm (iii)}
(q,c_0,c_2,\widetilde{\mu}_2,\mu_{\bf 11}^{\prime});\sin^2 \frac{1}{2} \theta_H \right] ,\cr
V_{\rm eff}^{\rm (iv-1)}(\theta_H)
&=- 4 \cdot \frac{1}{2} I\left[\widetilde{Q}_{\rm (iv-1)}
(q,c_0,m_B,M;\theta_H);1\right] , \cr
V_{\rm eff}^{\rm (iv-2)}(\theta_H)
&=-4I\left[\widetilde{Q}_{\rm (iv-2)}
(q,c_2,\mu_{\bf 11}^{\prime};\theta_H);1\right] ,\cr
V_{\rm eff}^{\rm (v)}(\theta_H)
&=-32I \left[Q_0\left(q,c_0'\right);\cos^2 \frac{1}{2} \theta_H \right] ,
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-M5-8"><label>(5.8)</label><tex-math notation="LaTeX" id="Equation139"><![CDATA[
\begin{align}
&Q_{\rm (ii)}(q) =
\frac{z_L}{q^2}
\left\{ \hat{F}_{c_0}^{++}\hat{F}_{c_0}^{-}
+\frac{\mu_1^2 \hat{F}_{c_0}^{-} \hat{F}_{c_0}^{-+} \hat{F}_{c_1}^{++} \hat{F}_{c_1}^{-+}}
{\mu_{\bf 11}^{2} (\hat{F}_{c_1}^{-+} )^2 + (\hat{F}_{c_1}^{++} )^2} \right\}^{-1}, \cr
&Q_{\rm (iii)}(q)=
\frac{z_L}{q^2}
\left\{ \hat{F}_{c_0}^{++}\hat{F}_{c_0}^{-}
+\frac{\widetilde{\mu}_2^2 \hat{F}_{c_0}^{++} \hat{F}_{c_0}^{+-}
\hat{F}_{c_2}^{-} \hat{F}_{c_2}^{+-}}
{\mu_{\bf 11}^{\prime 2} (\hat{F}_{c_2}^{+-} )^2 + (\hat{F}_{c_2}^{-} )^2} \right\}^{-1}, \cr
&Q_{\rm (iv-1)}(q)=
\frac{z_L}{q^2}
\left\{\hat{F}_{c_0}^{++}\hat{F}_{c_0}^{-}
-\frac{ i m_B^2 / k^2}{2(iqz_L^{-1}+M/k)}
\hat{F}_{c_0}^{-+}\hat{F}_{c_0}^{-}\right\}^{-1}, \cr
&\widetilde{Q}_{\rm (iv-1)}(q;\theta_H) =
(Q_{\rm (iv-1)}(q)+Q_{\rm (iv-1)}(q)^*)\sin^2 \frac{1}{2} \theta_H \cr
&\hspace{5.5cm}
+( Q_{\rm (iv-1)}(q)Q_{\rm (iv-1)}(q)^* )\sin^4 \frac{1}{2} \theta_H , \cr
&\widetilde{Q}_{\rm (iv-2)}(q,\theta_H)= \frac{z_L^2}{q^4}
\frac{h_1 (q, \theta_H)}{h_2 (q)} , \cr
&
h_1 = ( 1 - \mu_{\bf 11}^{\prime \, 2} )
\Bigg\{ 2\frac{q^2}{z_L}
(1 + \mu_{\bf 11}^{\prime \, 2}) \hat{F}_{c_2}^{++}\hat{F}_{c_2}^{-}
+ 2 \mu_{\bf 11}^{\prime 2}
+(1 - \mu_{\bf 11}^{\prime \, 2}) \sin^2\theta_H \Bigg\} \sin^2\theta_H , \cr
&
h_2 = \frac{z_L^2}{q^4} \mu_{\bf 11}^{\prime 4}
+2\mu_{\bf 11}^{\prime 4} \frac{z_L}{q^2} \hat{F}_{c_2}^{++}\hat{F}_{c_2}^{-}
+(1+\mu_{\bf 11}^{\prime 4})
(\hat{F}_{c_2}^{++}\hat{F}_{c_2}^{-} )^2 \cr
&\hspace{6cm}
+\mu_{\bf 11}^{\prime 2}
\Big\{ (\hat{F}_{c_2}^{-+}\hat{F}_{c_2}^{-} )^2
+ (\hat{F}_{c_2}^{+-}\hat{F}_{c_2}^{++} )^2 \Big\} .
\end{align}]]></tex-math></disp-formula></p>
<p>As <inline-formula><tex-math notation="LaTeX" id="ImEquation863"><![CDATA[$Q_{\rm (iv-1)}(q)$]]></tex-math></inline-formula> is not a real function, <inline-formula><tex-math notation="LaTeX" id="ImEquation864"><![CDATA[$V_{\rm eff}^{\rm (iv-1)}(\theta_H)$]]></tex-math></inline-formula> has been converted to the integral involving <inline-formula><tex-math notation="LaTeX" id="ImEquation865"><![CDATA[$\widetilde{Q}_{\rm (iv-1)}(q;\theta_H)$]]></tex-math></inline-formula>.</p>
<p>The Higgs mass <inline-formula><tex-math notation="LaTeX" id="ImEquation866"><![CDATA[$m_H$]]></tex-math></inline-formula> is determined, at the minimum <inline-formula><tex-math notation="LaTeX" id="ImEquation867"><![CDATA[$\theta_H = \theta_H^{\rm min}$]]></tex-math></inline-formula> of <inline-formula><tex-math notation="LaTeX" id="ImEquation868"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula>, by
<disp-formula id="ptx175-M5-9"><label>(5.9)</label><tex-math notation="LaTeX" id="Equation140"><![CDATA[
\begin{align}
&m_H^2 = \frac{1}{f_H^2}
\frac{d^2V_{\rm eff}(\theta_H)}{d\theta_H^2}
\bigg|_{\theta_H = \theta_H^{\rm min}}, \cr
&f_H = \frac{\sqrt{6}}{g} \sqrt{\frac{2\pi R_6 \, k}{z_L^3 -1} }
= \frac{\sqrt{6}}{g_w} \frac{k}{\sqrt{(1-z_L^{-1}) (z_L^3 -1)}}\cr
&g_w = \frac{e}{\sin \theta_W}
= g \sqrt{ \frac{k}{2\pi R_6 (1 - z_L^{-1})}} .
\end{align}]]></tex-math></disp-formula></p>
<p>Note that the 4D neutral Higgs field <inline-formula><tex-math notation="LaTeX" id="ImEquation869"><![CDATA[$H(x)$]]></tex-math></inline-formula> is related to <inline-formula><tex-math notation="LaTeX" id="ImEquation870"><![CDATA[$\phi_H(x)$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-33">2.33</xref>) by
<disp-formula id="ptx175-M5-10"><label>(5.10)</label><tex-math notation="LaTeX" id="Equation141"><![CDATA[
\begin{align}
\phi_H(x)= \theta_H f_H + H(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>We give results for the effective potential and the Higgs mass for typical sample values of the parameters. In the following calculation, we take into account the sixth-dimensional <inline-formula><tex-math notation="LaTeX" id="ImEquation871"><![CDATA[$n=0$]]></tex-math></inline-formula> modes of 6D bulk gauge bosons <inline-formula><tex-math notation="LaTeX" id="ImEquation872"><![CDATA[$A_M$]]></tex-math></inline-formula>, the third-generation <inline-formula><tex-math notation="LaTeX" id="ImEquation873"><![CDATA[$SO(11)$]]></tex-math></inline-formula> spinor and vector fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation874"><![CDATA[$\Psi_{\bf 32}^{\alpha=3}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation875"><![CDATA[$\Psi_{\bf 11}^{\beta=3}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation876"><![CDATA[$\Psi_{\bf 11}^{\prime\beta=3}$]]></tex-math></inline-formula>, and dark fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation877"><![CDATA[$\Psi_{\bf 32}^{\alpha=4}$]]></tex-math></inline-formula>. Contributions coming from the first and second generations of quarks and leptons are numerically negligible. Some of the parameters in the fermion sector are fixed by the observed masses of quarks and leptons in their approximate forms given in Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-23">4.23</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-31">4.31</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-40">4.40</xref>), and (<xref ref-type="disp-formula" rid="ptx175-M4-56">4.56</xref>).</p>
<p>The calculation algorithm is almost the same as in 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation878"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT given in <xref ref-type="sec" rid="SEC5">Sect. 5</xref> of Ref. [<xref ref-type="bibr" rid="B35">35</xref>]. We are interested in the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation879"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula> at the electroweak scale. Some of the relations derived in this paper are those at the GUT scale. The value of <inline-formula><tex-math notation="LaTeX" id="ImEquation880"><![CDATA[$\sin^2 \theta_W$]]></tex-math></inline-formula> evolves from <inline-formula><tex-math notation="LaTeX" id="ImEquation881"><![CDATA[$\frac{3}{8}$]]></tex-math></inline-formula> at the GUT scale to the observed value at the electroweak scale by the renormalization group equation (RGE). The RGE analysis of the coupling constants is beyond the scope of the current paper. In this paper we content ourselves with inserting the observed value of <inline-formula><tex-math notation="LaTeX" id="ImEquation882"><![CDATA[$\sin^2 \theta_W$]]></tex-math></inline-formula> into the formulas for <inline-formula><tex-math notation="LaTeX" id="ImEquation883"><![CDATA[$m_Z$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation884"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula>. As input parameters we take <inline-formula><tex-math notation="LaTeX" id="ImEquation885"><![CDATA[$\alpha_{\rm EM}^{-1}=127.916$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation886"><![CDATA[$\sin^2\theta_W=0.2312$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation887"><![CDATA[$m_Z=91.1876\,\mbox{GeV}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation888"><![CDATA[$m_t=171.17\,\mbox{GeV}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation889"><![CDATA[$m_b=4.18\,\mbox{GeV}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation890"><![CDATA[$m_\tau=1.776\,\mbox{GeV}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation891"><![CDATA[$m_{\nu_\tau}=0.1\,\mbox{eV}$]]></tex-math></inline-formula>. The evaluation is carried out in the following steps:</p>
<p><list list-type="simple">
<list-item><p>(1) We first pick the values for <inline-formula><tex-math notation="LaTeX" id="ImEquation892"><![CDATA[$z_L$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation893"><![CDATA[$\theta_H$]]></tex-math></inline-formula>. So far consistent sets of parameters have been found only for <inline-formula><tex-math notation="LaTeX" id="ImEquation894"><![CDATA[$30 \lesssim z_L \lesssim 40$]]></tex-math></inline-formula>.</p></list-item>
<list-item><p>(2) From <inline-formula><tex-math notation="LaTeX" id="ImEquation895"><![CDATA[$m_Z$]]></tex-math></inline-formula>, Eq. (<xref ref-type="disp-formula" rid="ptx175-M3-12">3.12</xref>), <inline-formula><tex-math notation="LaTeX" id="ImEquation896"><![CDATA[$k$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation897"><![CDATA[$m_{{\rm KK}_5}$]]></tex-math></inline-formula> are determined.</p></list-item>
<list-item><p>(3) <inline-formula><tex-math notation="LaTeX" id="ImEquation898"><![CDATA[$c_0$]]></tex-math></inline-formula> is fixed from <inline-formula><tex-math notation="LaTeX" id="ImEquation899"><![CDATA[$m_t$]]></tex-math></inline-formula> by Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-23">4.23</xref>).</p></list-item>
<list-item><p>(4) Some of the parameters in the brane interactions on the UV brane remain free. We take <inline-formula><tex-math notation="LaTeX" id="ImEquation900"><![CDATA[$c_1 =0$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation901"><![CDATA[$c_2 = -0.7$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation902"><![CDATA[$M = -10^{7}\,$]]></tex-math></inline-formula>GeV. These three parameters are kept fixed in the evaluation below. We temporarily assign a value for <inline-formula><tex-math notation="LaTeX" id="ImEquation903"><![CDATA[$\mu_{\bf 11} = \mu_{\bf 11}'$]]></tex-math></inline-formula>. Then, <inline-formula><tex-math notation="LaTeX" id="ImEquation904"><![CDATA[$\mu_1$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation905"><![CDATA[$\widetilde{\mu}_{2}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation906"><![CDATA[$m_B$]]></tex-math></inline-formula> are determined by Eqs. (<xref ref-type="disp-formula" rid="ptx175-M4-32">4.32</xref>), (<xref ref-type="disp-formula" rid="ptx175-M4-41">4.41</xref>), and (<xref ref-type="disp-formula" rid="ptx175-M4-56">4.56</xref>), or by
<disp-formula id="ptx175-M5-11"><label>(5.11)</label><tex-math notation="LaTeX" id="Equation142"><![CDATA[
\begin{align}
&\mu_{1}\simeq
\sqrt{\frac{1+2c_1}{1+2c_0}}z_L^{c_0-c_1}\frac{m_t}{m_b}
\mu_{\bf 11} , \cr
&
\widetilde{\mu}_{2}\simeq
\sqrt{\frac{1-2c_2}{1-2c_0}} \, z_L^{c_2-c_0} \frac{m_t}{m_\tau}
\mu_{\bf 11}^{\prime} , \cr
&m_B=\sqrt{- \frac{2Mm_t^2}{m_{\nu_\tau}}\frac{z_L^{2c_0+1}}{1+2c_0}} .
\end{align}]]></tex-math></disp-formula></p></list-item>
<list-item><p>(5) Given the value of the bulk mass parameter <inline-formula><tex-math notation="LaTeX" id="ImEquation907"><![CDATA[$c_0'$]]></tex-math></inline-formula> of the dark fermion multiplet, the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation908"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula> can be evaluated. We adjust the value of <inline-formula><tex-math notation="LaTeX" id="ImEquation909"><![CDATA[$c_0'$]]></tex-math></inline-formula> such that the minimum of <inline-formula><tex-math notation="LaTeX" id="ImEquation910"><![CDATA[$V_{\rm eff} (\theta_H)$]]></tex-math></inline-formula> is located at the initial value of <inline-formula><tex-math notation="LaTeX" id="ImEquation911"><![CDATA[$\theta_H$]]></tex-math></inline-formula>. In this manner a consistent parameter set has been obtained.</p></list-item>
<list-item><p>(6) The Higgs boson mass <inline-formula><tex-math notation="LaTeX" id="ImEquation912"><![CDATA[$m_H$]]></tex-math></inline-formula> is determined from (<xref ref-type="disp-formula" rid="ptx175-M5-9">5.9</xref>), which can be viewed as a function of <inline-formula><tex-math notation="LaTeX" id="ImEquation913"><![CDATA[$z_L$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation914"><![CDATA[$\theta_H$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation915"><![CDATA[$\mu^{\prime}_{\bf 11}$]]></tex-math></inline-formula>, namely <inline-formula><tex-math notation="LaTeX" id="ImEquation916"><![CDATA[$m_H (\theta_H, z_L,\mu^{\prime}_{\bf 11})$]]></tex-math></inline-formula>.</p></list-item>
<list-item><p>(7) By demanding that the resulting <inline-formula><tex-math notation="LaTeX" id="ImEquation917"><![CDATA[$m_H$]]></tex-math></inline-formula> is the observed value, namely <inline-formula><tex-math notation="LaTeX" id="ImEquation918"><![CDATA[$m_H (z_L, \theta_H) = 125.09 \pm 0.24 \,$]]></tex-math></inline-formula>GeV, <inline-formula><tex-math notation="LaTeX" id="ImEquation919"><![CDATA[$\mu^{\prime}_{\bf 11}$]]></tex-math></inline-formula> is determined with the given <inline-formula><tex-math notation="LaTeX" id="ImEquation920"><![CDATA[$\theta_H$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation921"><![CDATA[$z_L$]]></tex-math></inline-formula>.</p></list-item>
<list-item><p>(8) Take a different <inline-formula><tex-math notation="LaTeX" id="ImEquation922"><![CDATA[$\theta_H$]]></tex-math></inline-formula>. By repeating the above procedure one finds a new value for <inline-formula><tex-math notation="LaTeX" id="ImEquation923"><![CDATA[$\mu^{\prime}_{\bf 11}$]]></tex-math></inline-formula>.</p></list-item>
</list></p>
<p>The physical value of <inline-formula><tex-math notation="LaTeX" id="ImEquation924"><![CDATA[$\theta_H$]]></tex-math></inline-formula> needs to be determined, for instance, from observation of the <inline-formula><tex-math notation="LaTeX" id="ImEquation925"><![CDATA[$Z'$]]></tex-math></inline-formula> bosons (the first KK bosons <inline-formula><tex-math notation="LaTeX" id="ImEquation926"><![CDATA[$\gamma^{(1)}, Z^{(1)}, Z_R^{(1)}$]]></tex-math></inline-formula>) and their masses. In the <inline-formula><tex-math notation="LaTeX" id="ImEquation927"><![CDATA[$SO(5) \times U(1)$]]></tex-math></inline-formula> gauge&#x2013;Higgs electroweak unification many of the physical quantities depend on the value of <inline-formula><tex-math notation="LaTeX" id="ImEquation928"><![CDATA[$\theta_H$]]></tex-math></inline-formula>, but not on the details of the parameters in the theory. It has been shown that <inline-formula><tex-math notation="LaTeX" id="ImEquation929"><![CDATA[$m_{Z^{(1)}} \gtrsim 7\,$]]></tex-math></inline-formula>TeV (<inline-formula><tex-math notation="LaTeX" id="ImEquation930"><![CDATA[$m_{{\rm KK}_5} \gtrsim 8.5\,$]]></tex-math></inline-formula>TeV) to be consistent with the current LHC data [<xref ref-type="bibr" rid="B15">15</xref>,<xref ref-type="bibr" rid="B16">16</xref>].</p>
<p>The results for the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation931"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> in the <inline-formula><tex-math notation="LaTeX" id="ImEquation932"><![CDATA[$R_{\xi=0}$]]></tex-math></inline-formula> gauge are depicted in <xref ref-type="fig" rid="F1">Fig. 1</xref>, for which <inline-formula><tex-math notation="LaTeX" id="ImEquation933"><![CDATA[$z_L=35$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation934"><![CDATA[$\theta_H=0.15$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation935"><![CDATA[$m_H = 125.1\,$]]></tex-math></inline-formula>GeV. The bulk and brane mass parameters are chosen as
<disp-formula id="ptx175-M5-12"><label>(5.12)</label><tex-math notation="LaTeX" id="Equation143"><![CDATA[
\begin{align}
&z_L=35,\ \ \theta_H=0.15,\ \ m_H=125.12\ \mbox{GeV},
\nonumber\\
&c_0=0.3325,\ \ c_1=0,\ \ c_2=-0.7 ,\ \ c_0'=0.5224,
\nonumber\\
&\mu_1\simeq 11.18,\ \
\widetilde{\mu}_2\simeq 0.7091,\ \
\mu_{\bf 11}=\mu_{\bf 11}^{\prime}=0.108,
\nonumber\\
&m_B=1.145\times 10^{12}\ \mbox{GeV},\ \
M=-10^{7}\ \mbox{GeV}.
\end{align}]]></tex-math></disp-formula></p>
<fig id="F1" orientation="portrait" position="float"><label>Fig. 1.</label><caption><p>The effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation936"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> in the <inline-formula><tex-math notation="LaTeX" id="ImEquation937"><![CDATA[$R_{\xi=0}$]]></tex-math></inline-formula> gauge in which <inline-formula><tex-math notation="LaTeX" id="ImEquation938"><![CDATA[$z_L=35$]]></tex-math></inline-formula> and the minimum of <inline-formula><tex-math notation="LaTeX" id="ImEquation939"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> is located at <inline-formula><tex-math notation="LaTeX" id="ImEquation940"><![CDATA[$\theta_H=0.15$]]></tex-math></inline-formula>. For the top two figures, the blue solid line shows the effective potential with both gauge and fermion effects, and the red and green dashed lines represent the contributions of fermions and gauge fields, respectively. For the bottom left figure, the green solid line represents the total contributions of gauge fields, and the red and purple dashed lines represent <inline-formula><tex-math notation="LaTeX" id="ImEquation941"><![CDATA[$Z$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation942"><![CDATA[$W$]]></tex-math></inline-formula> boson contributions. In the bottom right figure the red solid line represents the total contribution of all fermions, the green dashed lines represent contributions of the top quark, the blue dashed line represents that of the neutrino sector-2, and the black dashed line represents the contribution of dark fermions. Contributions coming from other fermions are negligible.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" orientation="portrait" position="float" xlink:href="ptx175F1.tif"/></fig>
<p>The resultant <inline-formula><tex-math notation="LaTeX" id="ImEquation943"><![CDATA[$m_{{\rm KK}_5} = \pi k z_L^{-1}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation944"><![CDATA[$k$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation945"><![CDATA[$f_H$]]></tex-math></inline-formula> are given by
<disp-formula id="ptx175-M5-13"><label>(5.13)</label><tex-math notation="LaTeX" id="Equation144"><![CDATA[
\begin{align}
m_{\rm KK_5}\simeq 8.236\ \mbox{TeV},\ \
k\simeq 8.913\times 10^{4}\ \mbox{GeV},\ \
f_H\simeq 1.642\times 10^{3}\ \mbox{GeV}.
\end{align}]]></tex-math></disp-formula></p>
<p>In <xref ref-type="table" rid="T6">Table 6</xref>, we have tabulated various sets of the parameters which yield <inline-formula><tex-math notation="LaTeX" id="ImEquation946"><![CDATA[$m_H=125.1 \pm 0.1$]]></tex-math></inline-formula> GeV. As <inline-formula><tex-math notation="LaTeX" id="ImEquation947"><![CDATA[$\theta_H$]]></tex-math></inline-formula> increases, <inline-formula><tex-math notation="LaTeX" id="ImEquation948"><![CDATA[$m_{\rm KK}$]]></tex-math></inline-formula> decreases. One sees <inline-formula><tex-math notation="LaTeX" id="ImEquation949"><![CDATA[$\theta_H \lesssim 0.15$]]></tex-math></inline-formula> in order for <inline-formula><tex-math notation="LaTeX" id="ImEquation950"><![CDATA[$m_{\rm KK} > 8\,$]]></tex-math></inline-formula>TeV to be consistent with the current LHC data. We have observed that <inline-formula><tex-math notation="LaTeX" id="ImEquation951"><![CDATA[$\theta_H$]]></tex-math></inline-formula> cannot be too small in order to reproduce <inline-formula><tex-math notation="LaTeX" id="ImEquation952"><![CDATA[$m_H \sim 125 \,$]]></tex-math></inline-formula>GeV.</p>
<p><table-wrap id="T6" orientation="portrait" position="float"><label>Table 6.</label><caption><p>Parameter sets which give dynamical EW symmetry breaking with <inline-formula><tex-math notation="LaTeX" id="ImEquation953"><![CDATA[$m_H=125.1 \pm 0.1$]]></tex-math></inline-formula> GeV. Here <inline-formula><tex-math notation="LaTeX" id="ImEquation954"><![CDATA[$z_L=35$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation955"><![CDATA[$c_1=0.0, c_2=-0.7$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation956"><![CDATA[$M = - 10^7\,$]]></tex-math></inline-formula>GeV. <inline-formula><tex-math notation="LaTeX" id="ImEquation957"><![CDATA[$c_2$]]></tex-math></inline-formula> can be varied without affecting <inline-formula><tex-math notation="LaTeX" id="ImEquation958"><![CDATA[$m_{{\rm KK}_5}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation959"><![CDATA[$c_0$]]></tex-math></inline-formula>. For <inline-formula><tex-math notation="LaTeX" id="ImEquation960"><![CDATA[$\theta_H = 0.05$]]></tex-math></inline-formula>, for instance, a set of <inline-formula><tex-math notation="LaTeX" id="ImEquation961"><![CDATA[$c_2 = - 0.9$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation962"><![CDATA[$\mu_{\bf 11}=\mu_{\bf 11}^\ell = 0.1174$]]></tex-math></inline-formula> yields <inline-formula><tex-math notation="LaTeX" id="ImEquation963"><![CDATA[$m_H = 125.05\,$]]></tex-math></inline-formula>GeV, <inline-formula><tex-math notation="LaTeX" id="ImEquation964"><![CDATA[$c_0' = 0.6008$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation965"><![CDATA[$\widetilde{\mu}_2 = 0.4084$]]></tex-math></inline-formula>.</p></caption>
<table frame="hsides" rules="groups">
<thead align="left">
<tr>
<th align="left"><inline-formula><tex-math notation="LaTeX" id="ImEquation966"><![CDATA[$\theta_H$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation967"><![CDATA[$\mu_{\bf 11}=\mu_{\bf 11}^l$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation968"><![CDATA[$m_{{\rm KK}_5}$]]></tex-math></inline-formula> [TeV]</th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation969"><![CDATA[$c_0$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation970"><![CDATA[$c_0'$]]></tex-math></inline-formula></th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation971"><![CDATA[$m_H$]]></tex-math></inline-formula> [GeV]</th>
<th align="center"><inline-formula><tex-math notation="LaTeX" id="ImEquation972"><![CDATA[$\widetilde{\mu}_2$]]></tex-math></inline-formula></th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">0.05</td>
<td align="center">0.1730</td>
<td align="center">24.63</td>
<td align="center">0.3289</td>
<td align="center">0.5960</td>
<td align="center">125.17</td>
<td align="center">1.138</td>
</tr>
<tr>
<td align="left">0.06</td>
<td align="center">0.1590</td>
<td align="center">20.53</td>
<td align="center">0.3292</td>
<td align="center">0.5828</td>
<td align="center">125.07</td>
<td align="center">1.046</td>
</tr>
<tr>
<td align="left">0.07</td>
<td align="center">0.1480</td>
<td align="center">17.60</td>
<td align="center">0.3294</td>
<td align="center">0.5714</td>
<td align="center">125.13</td>
<td align="center">0.9736</td>
</tr>
<tr>
<td align="left">0.08</td>
<td align="center">0.1400</td>
<td align="center">15.40</td>
<td align="center">0.3297</td>
<td align="center">0.5625</td>
<td align="center">125.03</td>
<td align="center">0.9208</td>
</tr>
<tr>
<td align="left">0.09</td>
<td align="center">0.1330</td>
<td align="center">13.70</td>
<td align="center">0.3300</td>
<td align="center">0.5544</td>
<td align="center">125.08</td>
<td align="center">0.8746</td>
</tr>
<tr>
<td align="left">0.10</td>
<td align="center">0.1270</td>
<td align="center">12.33</td>
<td align="center">0.3303</td>
<td align="center">0.5471</td>
<td align="center">125.17</td>
<td align="center">0.8350</td>
</tr>
<tr>
<td align="left">0.11</td>
<td align="center">0.1225</td>
<td align="center">11.21</td>
<td align="center">0.3307</td>
<td align="center">0.5414</td>
<td align="center">125.05</td>
<td align="center">0.8052</td>
</tr>
<tr>
<td align="left">0.12</td>
<td align="center">0.1180</td>
<td align="center">10.28</td>
<td align="center">0.3311</td>
<td align="center">0.5356</td>
<td align="center">125.14</td>
<td align="center">0.7755</td>
</tr>
<tr>
<td align="left">0.13</td>
<td align="center">0.1145</td>
<td align="center">9.495</td>
<td align="center">0.3315</td>
<td align="center">0.5311</td>
<td align="center">125.06</td>
<td align="center">0.7523</td>
</tr>
<tr>
<td align="left">0.14</td>
<td align="center">0.1110</td>
<td align="center">8.821</td>
<td align="center">0.3320</td>
<td align="center">0.5264</td>
<td align="center">125.12</td>
<td align="center">0.7291</td>
</tr>
<tr>
<td align="left">0.15</td>
<td align="center">0.1080</td>
<td align="center">8.236</td>
<td align="center">0.3325</td>
<td align="center">0.5224</td>
<td align="center">125.12</td>
<td align="center">0.7091</td>
</tr>
<tr>
<td align="left">0.16</td>
<td align="center">0.1055</td>
<td align="center">7.725</td>
<td align="center">0.3331</td>
<td align="center">0.5192</td>
<td align="center">125.03</td>
<td align="center">0.6925</td>
</tr>
<tr>
<td align="left">0.17</td>
<td align="center">0.1030</td>
<td align="center">7.275</td>
<td align="center">0.3337</td>
<td align="center">0.5158</td>
<td align="center">125.02</td>
<td align="center">0.6759</td>
</tr>
<tr>
<td align="left">0.18</td>
<td align="center">0.1005</td>
<td align="center">6.875</td>
<td align="center">0.3343</td>
<td align="center">0.5125</td>
<td align="center">125.11</td>
<td align="center">0.6593</td>
</tr>
<tr>
<td align="left">0.19</td>
<td align="center">0.0985</td>
<td align="center">6.517</td>
<td align="center">0.3349</td>
<td align="center">0.5099</td>
<td align="center">125.05</td>
<td align="center">0.6459</td>
</tr>
<tr>
<td align="left">0.20</td>
<td align="center">0.0965</td>
<td align="center">6.195</td>
<td align="center">0.3356</td>
<td align="center">0.5073</td>
<td align="center">125.05</td>
<td align="center">0.6326</td>
</tr>
<tr>
<td align="left">0.21</td>
<td align="center">0.0945</td>
<td align="center">5.903</td>
<td align="center">0.3363</td>
<td align="center">0.5047</td>
<td align="center">125.21</td>
<td align="center">0.6192</td>
</tr>
<tr>
<td align="left">0.22</td>
<td align="center">0.0930</td>
<td align="center">5.639</td>
<td align="center">0.3371</td>
<td align="center">0.5030</td>
<td align="center">125.00</td>
<td align="center">0.6092</td>
</tr>
<tr>
<td align="left">0.23</td>
<td align="center">0.0910</td>
<td align="center">5.398</td>
<td align="center">0.3379</td>
<td align="center">0.5003</td>
<td align="center">125.17</td>
<td align="center">0.5959</td>
</tr>
<tr>
<td align="left">0.24</td>
<td align="center">0.0895</td>
<td align="center">5.177</td>
<td align="center">0.3387</td>
<td align="center">0.4986</td>
<td align="center">125.14</td>
<td align="center">0.5858</td>
</tr>
<tr>
<td align="left">0.25</td>
<td align="center">0.0880</td>
<td align="center">4.974</td>
<td align="center">0.3395</td>
<td align="center">0.4969</td>
<td align="center">125.16</td>
<td align="center">0.5758</td>
</tr>
<tr>
<td align="left">0.26</td>
<td align="center">0.0867</td>
<td align="center">4.786</td>
<td align="center">0.3404</td>
<td align="center">0.4956</td>
<td align="center">125.11</td>
<td align="center">0.5671</td>
</tr>
<tr>
<td align="left">0.27</td>
<td align="center">0.0855</td>
<td align="center">4.613</td>
<td align="center">0.3413</td>
<td align="center">0.4945</td>
<td align="center">125.04</td>
<td align="center">0.5590</td>
</tr>
<tr>
<td align="left">0.28</td>
<td align="center">0.0840</td>
<td align="center">4.452</td>
<td align="center">0.3423</td>
<td align="center">0.4928</td>
<td align="center">125.16</td>
<td align="center">0.5490</td>
</tr>
<tr>
<td align="left">0.29</td>
<td align="center">0.0830</td>
<td align="center">4.302</td>
<td align="center">0.3432</td>
<td align="center">0.4921</td>
<td align="center">125.05</td>
<td align="center">0.5423</td>
</tr>
<tr>
<td align="left">0.30</td>
<td align="center">0.0818</td>
<td align="center">4.163</td>
<td align="center">0.3442</td>
<td align="center">0.4911</td>
<td align="center">125.07</td>
<td align="center">0.5342</td>
</tr>
</tbody>
</table>
</table-wrap></p>
<p>In <xref ref-type="fig" rid="F2">Figure 2</xref>, <inline-formula><tex-math notation="LaTeX" id="ImEquation973"><![CDATA[$m_{{\rm KK}_5}$]]></tex-math></inline-formula> is plotted as a function of <inline-formula><tex-math notation="LaTeX" id="ImEquation974"><![CDATA[$\theta_H$]]></tex-math></inline-formula>. The data points in <xref ref-type="table" rid="T6">Table 6</xref> are fitted by
<disp-formula id="ptx175-M5-14"><label>(5.14)</label><tex-math notation="LaTeX" id="Equation145"><![CDATA[
\begin{align}
m_{{\rm KK}_5} \sim \frac{\alpha}{(\sin \theta_H)^{\beta}} ,
\alpha = 1.230\,{\rm TeV} , \beta = 1.000
\end{align}]]></tex-math></disp-formula>
in the range <inline-formula><tex-math notation="LaTeX" id="ImEquation975"><![CDATA[$0.05 < \theta_H < 0.30$]]></tex-math></inline-formula>. A similar relation has been found in the <inline-formula><tex-math notation="LaTeX" id="ImEquation976"><![CDATA[$SO(5) \times U(1)$]]></tex-math></inline-formula> gauge&#x2013;Higgs EW unification where <inline-formula><tex-math notation="LaTeX" id="ImEquation977"><![CDATA[$\alpha=1.352\,$]]></tex-math></inline-formula>TeV and <inline-formula><tex-math notation="LaTeX" id="ImEquation978"><![CDATA[$\beta = 0.786$]]></tex-math></inline-formula> [<xref ref-type="bibr" rid="B11">11</xref>,<xref ref-type="bibr" rid="B12">12</xref>]. The difference between the two is expected to originate from the different gauge group structure of the models and the different matter content. So far we have found consistent parameter sets only for <inline-formula><tex-math notation="LaTeX" id="ImEquation979"><![CDATA[$c_1=0$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation980"><![CDATA[$c_2 \le 0$]]></tex-math></inline-formula>.</p>
<fig id="F2" orientation="portrait" position="float"><label>Fig. 2.</label><caption><p>The Kaluza&#x2013;Klein mass scale <inline-formula><tex-math notation="LaTeX" id="ImEquation981"><![CDATA[$m_{\rm KK} = m_{{\rm KK}_5}$]]></tex-math></inline-formula> as a function of <inline-formula><tex-math notation="LaTeX" id="ImEquation982"><![CDATA[$\theta_H$]]></tex-math></inline-formula>. The red dots represent points in <xref ref-type="table" rid="T6">Table 6</xref> which reproduce <inline-formula><tex-math notation="LaTeX" id="ImEquation983"><![CDATA[$m_H=125.1\pm 0.1$]]></tex-math></inline-formula> GeV. The blue thick dashed line represents the fitting curve given in Eq. (<xref ref-type="disp-formula" rid="ptx175-M5-14">5.14</xref>). The black thin dashed line indicates <inline-formula><tex-math notation="LaTeX" id="ImEquation984"><![CDATA[$m_{\rm KK} (\theta_H)$]]></tex-math></inline-formula> found in the <inline-formula><tex-math notation="LaTeX" id="ImEquation985"><![CDATA[$SO(5) \times U(1)$]]></tex-math></inline-formula> gauge&#x2013;Higgs EW unification [<xref ref-type="bibr" rid="B11">11</xref>,<xref ref-type="bibr" rid="B12">12</xref>].</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" orientation="portrait" position="float" xlink:href="ptx175F2.tif"/></fig>
</sec>
<sec id="SEC6"><title>6. Summary and discussions</title>
<p>In this paper, we discussed 6D <inline-formula><tex-math notation="LaTeX" id="ImEquation986"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT in the 6D hybrid warped space. The GUT gauge symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation987"><![CDATA[$SO(11)$]]></tex-math></inline-formula> is reduced to the Pati&#x2013;Salam symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation988"><![CDATA[$G_{\rm PS}\ (=SU(2)_L\times SU(2)_R\times SU(4)_C)$]]></tex-math></inline-formula> by the orbifold BCs, and the symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation989"><![CDATA[$G_{\rm PS}$]]></tex-math></inline-formula> is spontaneously broken to the SM gauge symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation990"><![CDATA[$G_{\rm SM}\ (=SU(3)_C\times SU(2)_L\times U(1)_Y)$]]></tex-math></inline-formula> by the non-vanishing VEV of the 5D brane scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation991"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula> on the UV brane. Finally, the EW gauge symmetry <inline-formula><tex-math notation="LaTeX" id="ImEquation992"><![CDATA[$SU(2)_L\times U(1)_Y$]]></tex-math></inline-formula> is broken to <inline-formula><tex-math notation="LaTeX" id="ImEquation993"><![CDATA[$U(1)_{\rm EM}$]]></tex-math></inline-formula> by the Hosotani mechanism. The SM Higgs boson appears as a zero mode of the fifth-dimensional components of the <inline-formula><tex-math notation="LaTeX" id="ImEquation994"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge field.</p>
<p>The SM fermions are contained in <inline-formula><tex-math notation="LaTeX" id="ImEquation995"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation996"><![CDATA[$\Psi_{\bf 11}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation997"><![CDATA[$\Psi_{\bf 11}^{\prime}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation998"><![CDATA[$\chi_{{\bf 1}}$]]></tex-math></inline-formula>. The SM fermions as well as the <inline-formula><tex-math notation="LaTeX" id="ImEquation999"><![CDATA[$W$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1000"><![CDATA[$Z$]]></tex-math></inline-formula> bosons acquire masses via the Hosotani mechanism. In <xref ref-type="sec" rid="SEC4">Sect. 4</xref>, we showed that the quark and lepton mass spectra can be reproduced by choosing the parameters of the bulk masses and the brane interactions. In the 5D GHGUT in Ref. [<xref ref-type="bibr" rid="B35">35</xref>], there necessarily have arisen light exotic vector-like fermions accompanied by SM fermions, which is experimentally unacceptable. In contrast to the case of the 5D GHGUT, the non-SM fermions have masses of either <inline-formula><tex-math notation="LaTeX" id="ImEquation1001"><![CDATA[$O(m_{\rm KK_{6}})$]]></tex-math></inline-formula> or <inline-formula><tex-math notation="LaTeX" id="ImEquation1002"><![CDATA[$O(m_{\rm KK_{5}})$]]></tex-math></inline-formula> in the current 6D model. Thus the problem of light exotic fermions has been solved.</p>
<p>From the analysis of the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation1003"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> in <xref ref-type="sec" rid="SEC5">Sect. 5</xref>, we have seen that the EW symmetry is dynamically broken by the AB phase <inline-formula><tex-math notation="LaTeX" id="ImEquation1004"><![CDATA[$\theta_H$]]></tex-math></inline-formula>. The Higgs boson mass <inline-formula><tex-math notation="LaTeX" id="ImEquation1005"><![CDATA[$m_H = 125.1\,$]]></tex-math></inline-formula>GeV can be obtained for <inline-formula><tex-math notation="LaTeX" id="ImEquation1006"><![CDATA[$0.05 \lesssim \theta_H \lesssim 0.30$]]></tex-math></inline-formula>, namely for <inline-formula><tex-math notation="LaTeX" id="ImEquation1007"><![CDATA[$25\, {\rm TeV} \gtrsim m_{{\rm KK}_5} \gtrsim 4.1 \,$]]></tex-math></inline-formula>TeV.</p>
<p>In the present paper we have treated each generation of quarks and leptons independently. It is necessary to incorporate the flavor mixing in the gauge&#x2013;Higgs grand unification. The mixing arises from the brane interactions. For quarks and charged leptons the mixing would result from the matrix structure of various <inline-formula><tex-math notation="LaTeX" id="ImEquation1008"><![CDATA[$\kappa$]]></tex-math></inline-formula>s and <inline-formula><tex-math notation="LaTeX" id="ImEquation1009"><![CDATA[$\mu$]]></tex-math></inline-formula>s in the brane interactions in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-27">2.27</xref>) of <inline-formula><tex-math notation="LaTeX" id="ImEquation1010"><![CDATA[$\Psi_{\bf 32}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation1011"><![CDATA[$\Psi_{\bf 11}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation1012"><![CDATA[$\Phi_{\bf 32}$]]></tex-math></inline-formula>. On the other hand, the dominant mixing in the neutrino sector would arise from the matrix structure in the mass <inline-formula><tex-math notation="LaTeX" id="ImEquation1013"><![CDATA[$M^{\beta \beta'}$]]></tex-math></inline-formula> associated with the symplectic Majorana fields <inline-formula><tex-math notation="LaTeX" id="ImEquation1014"><![CDATA[$\chi_{\bf 1}$]]></tex-math></inline-formula> in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-26">2.26</xref>). The large mixing angles observed in the neutrino sector might be related to this special circumstance.</p>
<p>We would like to add a comment on the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation1015"><![CDATA[$V_{\rm eff}^6(\theta_H^6)$]]></tex-math></inline-formula> of the AB phase <inline-formula><tex-math notation="LaTeX" id="ImEquation1016"><![CDATA[$\theta_H^6$]]></tex-math></inline-formula> along the sixth dimension. From <xref ref-type="table" rid="T5">Table 5</xref>, not only the <inline-formula><tex-math notation="LaTeX" id="ImEquation1017"><![CDATA[$SO(5)/SO(4)$]]></tex-math></inline-formula> components of the fifth-dimensional gauge field <inline-formula><tex-math notation="LaTeX" id="ImEquation1018"><![CDATA[$A_y$]]></tex-math></inline-formula> but also those of the sixth-dimensional gauge field <inline-formula><tex-math notation="LaTeX" id="ImEquation1019"><![CDATA[$A_v$]]></tex-math></inline-formula> have zero modes in the absence of brane interactions. As in the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation1020"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> for the AB phase <inline-formula><tex-math notation="LaTeX" id="ImEquation1021"><![CDATA[$\theta_H$]]></tex-math></inline-formula> along the fifth dimension, the <inline-formula><tex-math notation="LaTeX" id="ImEquation1022"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk gauge bosons <inline-formula><tex-math notation="LaTeX" id="ImEquation1023"><![CDATA[$A_M$]]></tex-math></inline-formula> and the <inline-formula><tex-math notation="LaTeX" id="ImEquation1024"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation1025"><![CDATA[$\Psi_{\bf 32}^{\alpha}$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation1026"><![CDATA[$\Psi_{\bf 11}^{\beta}$]]></tex-math></inline-formula>, and <inline-formula><tex-math notation="LaTeX" id="ImEquation1027"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula>, contribute to <inline-formula><tex-math notation="LaTeX" id="ImEquation1028"><![CDATA[$V_{\rm eff}^6(\theta_H^6)$]]></tex-math></inline-formula>. If there were only gauge fields, it can be shown that the minimum of <inline-formula><tex-math notation="LaTeX" id="ImEquation1029"><![CDATA[$V_{\rm eff}^6(\theta_H^6)$]]></tex-math></inline-formula> would be located at <inline-formula><tex-math notation="LaTeX" id="ImEquation1030"><![CDATA[$\theta_H^6=\frac{1}{2} \pi \ ({\rm mod}\,\pi)$]]></tex-math></inline-formula>. In other words, the EW symmetry would be broken by the sixth-dimensional AB phase <inline-formula><tex-math notation="LaTeX" id="ImEquation1031"><![CDATA[$\theta_6$]]></tex-math></inline-formula> and the <inline-formula><tex-math notation="LaTeX" id="ImEquation1032"><![CDATA[$W$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1033"><![CDATA[$Z$]]></tex-math></inline-formula> bosons would acquire masses of <inline-formula><tex-math notation="LaTeX" id="ImEquation1034"><![CDATA[$O(m_{\rm KK_{6}})$]]></tex-math></inline-formula>. The situation is similar to that in the effective potential <inline-formula><tex-math notation="LaTeX" id="ImEquation1035"><![CDATA[$V_{\rm eff}(\theta_H)$]]></tex-math></inline-formula> in the 5D <inline-formula><tex-math notation="LaTeX" id="ImEquation1036"><![CDATA[$SO(11)$]]></tex-math></inline-formula> GHGUT [<xref ref-type="bibr" rid="B32">32</xref>,<xref ref-type="bibr" rid="B35">35</xref>]. Further, the contribution from the <inline-formula><tex-math notation="LaTeX" id="ImEquation1037"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk spinor fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation1038"><![CDATA[$\Psi_{\bf 32}^{\alpha}$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation1039"><![CDATA[$V_{\rm eff}^6(\theta_H^6)$]]></tex-math></inline-formula> has little <inline-formula><tex-math notation="LaTeX" id="ImEquation1040"><![CDATA[$\theta_H^6$]]></tex-math></inline-formula> dependence, as is inferred from <xref ref-type="table" rid="T2">Table 2</xref>. The <inline-formula><tex-math notation="LaTeX" id="ImEquation1041"><![CDATA[$\theta_H^6$]]></tex-math></inline-formula>-dependent contributions coming from a set of <inline-formula><tex-math notation="LaTeX" id="ImEquation1042"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk vector fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation1043"><![CDATA[$\Psi_{\bf 11}^{\beta}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1044"><![CDATA[$\Psi_{\bf 11}^{\prime\beta}$]]></tex-math></inline-formula> almost cancel each other, as can be deduced from <xref ref-type="table" rid="T3">Table 3</xref>. Hence, with the minimal fermion content presented in this paper the minimum of <inline-formula><tex-math notation="LaTeX" id="ImEquation1045"><![CDATA[$V_{\rm eff}^6(\theta_H^6)$]]></tex-math></inline-formula> would be located at around <inline-formula><tex-math notation="LaTeX" id="ImEquation1046"><![CDATA[$\theta_H^6=\pi/2\ ({\rm mod}\,\pi)$]]></tex-math></inline-formula>. This undesirable result can be avoided by introducing additional <inline-formula><tex-math notation="LaTeX" id="ImEquation1047"><![CDATA[$SO(11)$]]></tex-math></inline-formula> bulk vector fermions <inline-formula><tex-math notation="LaTeX" id="ImEquation1048"><![CDATA[$\Psi_{\bf 11}^{\beta}$]]></tex-math></inline-formula> with appropriate boundary conditions.</p>
<p>The phase <inline-formula><tex-math notation="LaTeX" id="ImEquation1049"><![CDATA[$\theta_H$]]></tex-math></inline-formula> plays a crucial role in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1050"><![CDATA[$SO(11)$]]></tex-math></inline-formula> gauge&#x2013;Higgs grand unification. It is found that the 5D KK mass scale <inline-formula><tex-math notation="LaTeX" id="ImEquation1051"><![CDATA[$m_{{\rm KK}_5}$]]></tex-math></inline-formula> is determined by <inline-formula><tex-math notation="LaTeX" id="ImEquation1052"><![CDATA[$\theta_H$]]></tex-math></inline-formula> in a very good approximation by the formula in Eq. (<xref ref-type="disp-formula" rid="ptx175-M5-14">5.14</xref>). Similar relations are expected for the 4D Higgs cubic and quartic couplings as in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1053"><![CDATA[$SO(5) \times U(1)$]]></tex-math></inline-formula> gauge&#x2013;Higgs EW unification. For <inline-formula><tex-math notation="LaTeX" id="ImEquation1054"><![CDATA[$m_{{\rm KK}_5} (\theta_H)$]]></tex-math></inline-formula> we have recognized the difference between the gauge&#x2013;Higgs EW and grand unification. We need experimental data to find which one is preferred.</p>
<p>We will come back to these issues in the near future.</p>
</sec>
</body>
<back>
<ack>
<title>Acknowledgments</title>
<p>We would like to thank Hisaki Hatanaka for critical reading of the manuscript and valuable remarks. This work was supported in part by Japan Society for the Promotion of Science Grant-in-Aid for Scientific Research No. 15K05052.</p>
</ack>
<sec><title>Funding</title>
<p>Open Access funding: SCOAP<inline-formula><tex-math notation="LaTeX" id="ImEquation1055"><![CDATA[$^3$]]></tex-math></inline-formula>.</p>
</sec>
<sec id="SECA"><title>Appendix A. Basics for 6D Kaluza&#x2013;Klein expansion</title>
<p>We present KK mode expansions in 6D hybrid warped space with the metric given in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-1">2.1</xref>).</p>
<sec id="SECA.1"><title>A.1. 6D gauge fields</title>
<p>The equations of motion for free gauge fields in the 6D hybrid warped space are given by
<disp-formula id="ptx175-MA-1"><label>(A.1)</label><tex-math notation="LaTeX" id="Equation146"><![CDATA[
\begin{align}
&\Big\{ \hat \eta^{\lambda \rho} \big(\Box+k^2 {\cal P}_5 +\partial_v^2 \big)-
\Big( 1-\frac{1}{\xi} \Big) \partial^\lambda \partial^\rho \Big\} A_\rho = 0 , \cr
&
\left(\Box+\xi k^2{\cal P}_z+\partial_v^2\right)A_z = 0,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1056"><![CDATA[${\cal P}_5$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1057"><![CDATA[${\cal P}_z$]]></tex-math></inline-formula> are defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-M2-32">2.32</xref>). These <inline-formula><tex-math notation="LaTeX" id="ImEquation1058"><![CDATA[${\cal P}_5$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1059"><![CDATA[${\cal P}_z$]]></tex-math></inline-formula> differ from <inline-formula><tex-math notation="LaTeX" id="ImEquation1060"><![CDATA[${\cal P}_4$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1061"><![CDATA[${\cal P}_z$]]></tex-math></inline-formula> in the 5d case. Basis mode functions in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1062"><![CDATA[$z$]]></tex-math></inline-formula>-coordinate in the 6d case are given, except for zero modes, by
<disp-formula id="ptx175-MA-2"><label>(A.2)</label><tex-math notation="LaTeX" id="Equation147"><![CDATA[
\begin{align}
&C(z;\lambda)
= +\frac{\pi}{2}\lambda z^{3/2} z_L^{1/2} F_{\frac{3}{2}, \frac{1}{2}}(\lambda z, \lambda z_L) , \cr
&S(z;\lambda)
= -\frac{\pi}{2}\lambda z^{3/2} z_L^{1/2} F_{\frac{3}{2}, \frac{3}{2}}(\lambda z, \lambda z_L) , \cr
&C'(z;\lambda)
= + \frac{\pi}{2}\lambda^2 z^{3/2} z_L^{1/2} F_{\frac{1}{2},\frac{1}{2}}(\lambda z, \lambda z_L) , \cr
&S'(z;\lambda)
= -\frac{\pi}{2} \lambda^2 z^{3/2} z_L^{1/2} F_{\frac{1}{2},\frac{3}{2}}(\lambda z, \lambda z_L) ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1063"><![CDATA[$F_{\alpha,\beta}(u,v) = J_\alpha (u) Y_\beta(v)-Y_\alpha(u) J_\beta(v)$]]></tex-math></inline-formula>. They can be expressed as
<disp-formula id="ptx175-MA-3"><label>(A.3)</label><tex-math notation="LaTeX" id="Equation148"><![CDATA[
\begin{align}
&C(z;\lambda)
= z \cos \lambda (z-z_L) - \frac{1}{\lambda} \sin \lambda (z-z_L) , \cr
&S(z;\lambda)
= \Big( z + \frac{1}{\lambda^2 z_L} \Big) \sin \lambda (z-z_L)
+ \Big( \frac{1}{\lambda} - \frac{z}{\lambda z_L} \Big) \cos \lambda (z-z_L) , \cr
&C'(z;\lambda)
= - \lambda z \sin \lambda (z-z_L) , \cr
&S'(z;\lambda)
= \lambda z \cos \lambda (z-z_L) + \frac{z}{z_L} \sin \lambda (z-z_L) ,
\end{align}]]></tex-math></disp-formula>
and satisfy
<disp-formula id="ptx175-MA-4"><label>(A.4)</label><tex-math notation="LaTeX" id="Equation149"><![CDATA[
\begin{align}
&{\cal P}_5 \begin{pmatrix} C \cr S \end{pmatrix} =
z^2 \frac{d}{dz} \frac{1}{z^2} \frac{d}{dz}
\begin{pmatrix} C \cr S \end{pmatrix} =
- \lambda^2 \begin{pmatrix} C \cr S \end{pmatrix} , \cr
&{\cal P}_z \begin{pmatrix} C' \cr S' \end{pmatrix} =
\frac{d}{dz} z^2 \frac{d}{dz} \frac{1}{z^2}
\begin{pmatrix} C' \cr S' \end{pmatrix} =
- \lambda^2 \begin{pmatrix} C' \cr S' \end{pmatrix} , \cr
&C(z_L; \lambda) = z_L , C' (z_L; \lambda) = 0 ,
S(z_L; \lambda) = 0 , S' (z_L; \lambda) = \lambda z_L , \cr
&C S' - S C' = \lambda z^2 .
\end{align}]]></tex-math></disp-formula></p>
<p>These basis functions have been employed in the text. We note that the basis functions for fermions, <inline-formula><tex-math notation="LaTeX" id="ImEquation1064"><![CDATA[$C_{L/R} (z; \lambda, c)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1065"><![CDATA[$S_{L/R} (z; \lambda, c)$]]></tex-math></inline-formula>, remain the same as in the 5d case defined in Ref.[<xref ref-type="bibr" rid="B35">35</xref>].</p>
</sec>
<sec id="SECA.2"><title>A.2. 6D scalar fields</title>
<p>The action of a massless 6D scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation1066"><![CDATA[$\Phi(x,y,v)$]]></tex-math></inline-formula> is given by
<disp-formula id="ptx175-MA-5"><label>(A.5)</label><tex-math notation="LaTeX" id="Equation150"><![CDATA[
\begin{align}
S_{\rm bulk}^{\rm scalar}=
\int d^6x\sqrt{-\mbox{det}\,G} \, G^{MN}
\partial_M\Phi(x,y,v)^\dagger \partial_N\Phi(x,y,v),
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1067"><![CDATA[$M,N=0,1,2,3,5,6$]]></tex-math></inline-formula>. The equation of motion for the scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation1068"><![CDATA[$\Phi(x,z,v)$]]></tex-math></inline-formula> is given by
<disp-formula id="ptx175-MA-6"><label>(A.6)</label><tex-math notation="LaTeX" id="Equation151"><![CDATA[
\begin{align}
z^2\left\{\Box_4+\partial_v^2
+k^2z^4\frac{\partial}{\partial z}\frac{1}{z^4}\frac{\partial}{\partial z}
\right\}\Phi(x,z,v)=0.
\end{align}]]></tex-math></disp-formula></p>
<p>It follows that <inline-formula><tex-math notation="LaTeX" id="ImEquation1069"><![CDATA[$\phi (x,z,v) = z^{-\nu} \Phi(x,z,v)$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation1070"><![CDATA[$\nu = \frac{5}{2}$]]></tex-math></inline-formula>) satisfies
<disp-formula id="ptx175-MA-7"><label>(A.7)</label><tex-math notation="LaTeX" id="Equation152"><![CDATA[
\begin{align}
\left\{\Box_4+\partial_v^2
+k^2\left(
\frac{\partial^2}{\partial z^2}
+\frac{1}{z}\frac{\partial}{\partial z}
-\frac{\nu^2}{z^2}\right)
\right\}\phi(x,z,v)=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>The Bessel equation is given by
<disp-formula id="ptx175-UM1"><tex-math notation="LaTeX" id="Equation153"><![CDATA[
\begin{align}
&\left(\frac{d^2}{dz^2}+\frac{1}{z}\frac{d}{dz}
+1-\frac{\nu^2}{z^2}\right)u(z)=0 ,
\nonumber
\end{align}]]></tex-math></disp-formula>
whose solutions are given by the Bessel functions <inline-formula><tex-math notation="LaTeX" id="ImEquation1071"><![CDATA[$J_\nu(z)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1072"><![CDATA[$Y_\nu(z)$]]></tex-math></inline-formula>. Hence, a mode function can be written as <inline-formula><tex-math notation="LaTeX" id="ImEquation1073"><![CDATA[$\phi(x,z,v) = [\alpha J_\nu (\lambda z) + \beta Y_\nu (\lambda z)] \phi_\lambda(x,v)$]]></tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX" id="ImEquation1074"><![CDATA[$\phi_\lambda(x,v)$]]></tex-math></inline-formula> satisfies
<disp-formula id="ptx175-MA-8"><label>(A.8)</label><tex-math notation="LaTeX" id="Equation154"><![CDATA[
\begin{align}
\left\{\Box_4+\partial_v^2-k^2\lambda^2\right\}\phi_\lambda(x, v)=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>The orbifold boundary condition for the scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation1075"><![CDATA[$\phi$]]></tex-math></inline-formula> is given by
<disp-formula id="ptx175-MA-9"><label>(A.9)</label><tex-math notation="LaTeX" id="Equation155"><![CDATA[
\begin{align}
&\Phi(x,y_j-y,v_j-v)=P_j \, \Phi(x,y_j+y,v_j+v) .
\end{align}]]></tex-math></disp-formula></p>
<p>In the <inline-formula><tex-math notation="LaTeX" id="ImEquation1076"><![CDATA[$y$]]></tex-math></inline-formula>-coordinate, <inline-formula><tex-math notation="LaTeX" id="ImEquation1077"><![CDATA[$\phi(x,y,v) = e^{- \nu\sigma(y)} \Phi (x,y,v)$]]></tex-math></inline-formula> satisfies the same boundary condition as Eq.(<xref ref-type="disp-formula" rid="ptx175-MA-9">A.9</xref>), and Eq.(<xref ref-type="disp-formula" rid="ptx175-MA-6">A.6</xref>) becomes
<disp-formula id="ptx175-MA-10"><label>(A.10)</label><tex-math notation="LaTeX" id="Equation156"><![CDATA[
\begin{align}
&
\left\{\Box_4+\partial_v^2+e^{\frac{1}{2}\sigma}
\left(\partial_y^2+\nu\sigma''-\nu^2\sigma'{}^2\right)
\right\}\phi=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>Due caution must be taken as <inline-formula><tex-math notation="LaTeX" id="ImEquation1078"><![CDATA[$\sigma '' (y) = 2k \{ \delta_{2L_5} (y) - \delta_{2L_5} (y-L_5) \}$]]></tex-math></inline-formula>. If <inline-formula><tex-math notation="LaTeX" id="ImEquation1079"><![CDATA[$\phi$]]></tex-math></inline-formula> is parity even in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1080"><![CDATA[$y$]]></tex-math></inline-formula>-coordinate, the Neumann (<inline-formula><tex-math notation="LaTeX" id="ImEquation1081"><![CDATA[$N$]]></tex-math></inline-formula>) condition becomes, in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1082"><![CDATA[$z$]]></tex-math></inline-formula>-coordinate (<inline-formula><tex-math notation="LaTeX" id="ImEquation1083"><![CDATA[$1 \le z \le z_L$]]></tex-math></inline-formula>),
<disp-formula id="ptx175-MA-11"><label>(A.11)</label><tex-math notation="LaTeX" id="Equation157"><![CDATA[
\begin{align}
N: \left( \frac{\partial}{\partial z} + \frac{\nu}{z} \right) \phi = 0 \quad
{\rm at~} z = 1^+ ~{\rm or}~ z_L^- ,
\end{align}]]></tex-math></disp-formula>
as can be confirmed by integrating Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-10">A.10</xref>) over <inline-formula><tex-math notation="LaTeX" id="ImEquation1084"><![CDATA[$y$]]></tex-math></inline-formula> from <inline-formula><tex-math notation="LaTeX" id="ImEquation1085"><![CDATA[$-\varepsilon$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation1086"><![CDATA[$+\varepsilon$]]></tex-math></inline-formula> (or from <inline-formula><tex-math notation="LaTeX" id="ImEquation1087"><![CDATA[$L_5 -\varepsilon$]]></tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX" id="ImEquation1088"><![CDATA[$L_5 +\varepsilon$]]></tex-math></inline-formula>). If <inline-formula><tex-math notation="LaTeX" id="ImEquation1089"><![CDATA[$\phi$]]></tex-math></inline-formula> is parity odd in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1090"><![CDATA[$y$]]></tex-math></inline-formula>-coordinate, it satisfies the Dirichlet (<inline-formula><tex-math notation="LaTeX" id="ImEquation1091"><![CDATA[$D$]]></tex-math></inline-formula>) condition <inline-formula><tex-math notation="LaTeX" id="ImEquation1092"><![CDATA[$\phi = 0$]]></tex-math></inline-formula> at <inline-formula><tex-math notation="LaTeX" id="ImEquation1093"><![CDATA[$z=1$]]></tex-math></inline-formula> or <inline-formula><tex-math notation="LaTeX" id="ImEquation1094"><![CDATA[$z_L$]]></tex-math></inline-formula>.</p>
<p>(i) <inline-formula><tex-math notation="LaTeX" id="ImEquation1095"><![CDATA[$(P_0=P_1,P_2=P_3)=(+,+)$]]></tex-math></inline-formula></p>
<p>The 6D scalar field <inline-formula><tex-math notation="LaTeX" id="ImEquation1096"><![CDATA[$\Phi(x,y,v)$]]></tex-math></inline-formula> is expanded as
<disp-formula id="ptx175-MA-12"><label>(A.12)</label><tex-math notation="LaTeX" id="Equation158"><![CDATA[
\begin{align}
&\Phi(x,y,v)= e^{\nu \sigma (y) } \left\{
\sum_{n=0}^{\infty}\widetilde{\phi}_n^C(x,y)f_n^C(v)
+\sum_{n=1}^{\infty}\widetilde{\phi}_n^S(x,y)f_n^S(v) \right\}\!,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1097"><![CDATA[$\widetilde{\phi}_n^C(x,y)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1098"><![CDATA[$\widetilde{\phi}_n^S(x,y)$]]></tex-math></inline-formula> satisfy
<disp-formula id="ptx175-MA-13"><label>(A.13)</label><tex-math notation="LaTeX" id="Equation159"><![CDATA[
\begin{align}
&
\left\{
\begin{array}{l}
\widetilde{\phi}_n^C(x,-y)=\widetilde{\phi}_n^C(x,y) , \cr
\widetilde{\phi}_n^C(x,L_5-y)=\widetilde{\phi}_n^C(x,L_5+y) , \\
\end{array}
\right. \cr
&
\left\{
\begin{array}{l}
\widetilde{\phi}_n^S(x,-y)=-\widetilde{\phi}_n^S(x,y) , \cr
\widetilde{\phi}_n^S(x,L_5-y)=-\widetilde{\phi}_n^S(x,L_5+y) , \\
\end{array}
\right.
\end{align}]]></tex-math></disp-formula>
and Fourier modes <inline-formula><tex-math notation="LaTeX" id="ImEquation1099"><![CDATA[$f_n^C(v)$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1100"><![CDATA[$f_n^S(v)$]]></tex-math></inline-formula> are given by
<disp-formula id="ptx175-MA-14"><label>(A.14)</label><tex-math notation="LaTeX" id="Equation160"><![CDATA[
\begin{align}
&
f_n^C (v) = \begin{cases} \displaystyle \frac{1}{\sqrt{2\pi R_6}} & (n=0) , \cr
\displaystyle
\frac{1}{\sqrt{\pi R_6}} \, \cos \frac{nv}{R_6} &(n\ge 1) , \end{cases} \cr
&
f_n^S (v) =\frac{1}{\sqrt{\pi R_6}} \, \sin \frac{nv}{R_6} (n\ge 1) .
\end{align}]]></tex-math></disp-formula>
<inline-formula><tex-math notation="LaTeX" id="ImEquation1101"><![CDATA[$\widetilde{\phi}_n^C$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1102"><![CDATA[$\widetilde{\phi}_n^S$]]></tex-math></inline-formula> satisfy the Neumann and Dirichlet conditions at <inline-formula><tex-math notation="LaTeX" id="ImEquation1103"><![CDATA[$z=1, z_L$]]></tex-math></inline-formula>, respectively. There is a zero mode for <inline-formula><tex-math notation="LaTeX" id="ImEquation1104"><![CDATA[$\widetilde{\phi}_n^C$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-MA-15"><label>(A.15)</label><tex-math notation="LaTeX" id="Equation161"><![CDATA[
\begin{align}
\lambda_0^C = 0 ,
\widetilde \phi_n^C(x, z) = \phi_{n,0}^C(x) z^{- \nu} ,
\end{align}]]></tex-math></disp-formula>
which corresponds to a mode constant in the fifth dimension. Other modes can be written as
<disp-formula id="ptx175-MA-16"><label>(A.16)</label><tex-math notation="LaTeX" id="Equation162"><![CDATA[
\begin{align}
&\widetilde{\phi}_n^{C,S} (x, z) = Z_\nu(\lambda z)\phi_{m,\lambda}^{C,S}(x) , \cr
&Z_\nu(\lambda z)=
\alpha_{n,\lambda}^{C,S}J_\nu(\lambda z)
+\beta_{n,\lambda}^{C,S}Y_\nu(\lambda z) .
\end{align}]]></tex-math></disp-formula></p>
<p>Eigenvalues <inline-formula><tex-math notation="LaTeX" id="ImEquation1105"><![CDATA[$\lambda >0$]]></tex-math></inline-formula> and the relative coefficient <inline-formula><tex-math notation="LaTeX" id="ImEquation1106"><![CDATA[$\alpha/\beta$]]></tex-math></inline-formula> are determined by the boundary conditions. Noting a recursion relation
<disp-formula id="ptx175-UM2"><tex-math notation="LaTeX" id="Equation163"><![CDATA[\[
\bigg(\frac{d}{dz}+\frac{\nu}{z}\bigg) Z_\nu(\lambda z)
=\lambda Z_{\nu-1}(\lambda z) ,
\]]]></tex-math></disp-formula>
one finds eigenvalues <inline-formula><tex-math notation="LaTeX" id="ImEquation1107"><![CDATA[$\big\{ \lambda_\ell^{C,S} \big\}$]]></tex-math></inline-formula> (<inline-formula><tex-math notation="LaTeX" id="ImEquation1108"><![CDATA[$\ell \ge 1$]]></tex-math></inline-formula>) from
<disp-formula id="ptx175-MA-17"><label>(A.17)</label><tex-math notation="LaTeX" id="Equation164"><![CDATA[
\begin{align}
\widetilde{\phi}_n^C &:
Z_{\nu -1} (\lambda_\ell^C)= Z_{\nu - 1} (\lambda_\ell^C z_L) = 0 , \cr
\widetilde{\phi}_n^S &:
Z_{\nu } (\lambda_\ell^S)= Z_{\nu} (\lambda_\ell^S z_L) = 0 .
\end{align}]]></tex-math></disp-formula></p>
<p>Thus <inline-formula><tex-math notation="LaTeX" id="ImEquation1109"><![CDATA[$\Phi$]]></tex-math></inline-formula> is expanded as
<disp-formula id="ptx175-MA-18"><label>(A.18)</label><tex-math notation="LaTeX" id="Equation165"><![CDATA[
\begin{align}
&\Phi(x,z,v)= e^{\nu \sigma (y) } \left\{
\sum_{n=0}^{\infty} \sum_{\ell=0}^\infty
{\phi}_{n, \ell}^C(x) Z_{\nu} (\lambda_\ell^C z) f_n^C(v)\right. \cr
&\hspace{3.7cm}\left.
+\sum_{n=1}^{\infty} \sum_{\ell=1}^\infty
{\phi}_{n, \ell} ^S(x) Z_{\nu} (\lambda_\ell^S z) f_n^S(v) \right\}\!,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1110"><![CDATA[$\widetilde{\phi}_{n, \ell}^{C,S} (x)$]]></tex-math></inline-formula> satisfies
<disp-formula id="ptx175-MA-19"><label>(A.19)</label><tex-math notation="LaTeX" id="Equation166"><![CDATA[
\begin{align}
&\Big\{\Box_4 - \big(m_{n,\ell}^{C,S} \big)^2 \Big\} \, \phi_{n,\ell}^{C,S}(x)=0 , \cr
&\big(m_{n,\ell}^{C,S} \big)^2 = \frac{n^2}{R_6^2} + \big( k \lambda_\ell^{C,S} \big)^2 .
\end{align}]]></tex-math></disp-formula></p>
<p>There is one massless mode <inline-formula><tex-math notation="LaTeX" id="ImEquation1111"><![CDATA[$\phi_{0,0}^C(x)$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation1112"><![CDATA[$m_{0,0}^C = 0$]]></tex-math></inline-formula>.</p>
<p>(ii) <inline-formula><tex-math notation="LaTeX" id="ImEquation1113"><![CDATA[$(P_0=P_1,P_2=P_3)=(-,-)$]]></tex-math></inline-formula></p>
<p>The expansion is given by
<disp-formula id="ptx175-MA-20"><label>(A.20)</label><tex-math notation="LaTeX" id="Equation167"><![CDATA[
\begin{align}
&\Phi(x,z,v)= e^{\nu \sigma (y) } \left\{
\sum_{n=0}^{\infty} \sum_{\ell=1}^\infty
{\phi}_{n, \ell}^S(x) Z_{\nu} (\lambda_\ell^S z) f_n^C(v)\right.\cr
&\left.\hspace{3.7cm}
+\sum_{n=1}^{\infty} \sum_{\ell=0}^\infty
{\phi}_{n, \ell} ^C(x) Z_{\nu} (\lambda_\ell^C z) f_n^S(v) \right\}\!.
\end{align}]]></tex-math></disp-formula></p>
<p>There is no massless mode.</p>
<p>(iii) <inline-formula><tex-math notation="LaTeX" id="ImEquation1114"><![CDATA[$(P_0=P_1,P_2=P_3)=(+,-)$]]></tex-math></inline-formula>
<disp-formula id="ptx175-MA-21"><label>(A.21)</label><tex-math notation="LaTeX" id="Equation168"><![CDATA[
\begin{align}
&\Phi(x,z,v)= e^{\nu \sigma (y) } \left\{
\sum_{n=0}^{\infty} \sum_{\ell=0}^\infty
{\phi}_{n, \ell}^C(x) Z_{\nu} (\lambda_\ell^C z)g_{n+\frac{1}{2}}^C(v)\right.\cr
&\left.\hspace{3.7cm}
+\sum_{n=0}^{\infty} \sum_{\ell=1}^\infty
{\phi}_{n, \ell} ^S(x) Z_{\nu} (\lambda_\ell^S z) g_{n+\frac{1}{2}}^S(v) \right\}\! ,
\end{align}]]></tex-math></disp-formula>
where
<disp-formula id="ptx175-MA-22"><label>(A.22)</label><tex-math notation="LaTeX" id="Equation169"><![CDATA[
\begin{align}
&g_{n+\frac{1}{2}}^C(v) = \frac{1}{\sqrt{\pi R_6}} \, \cos \frac{(n+\frac{1}{2})v}{R_6} ,
g_{n+\frac{1}{2}}^S(v) = \frac{1}{\sqrt{\pi R_6}} \, \sin \frac{(n+\frac{1}{2})v}{R_6} .
\end{align}]]></tex-math></disp-formula></p>
<p>Note that the mass spectrum is given by
<disp-formula id="ptx175-MA-23"><label>(A.23)</label><tex-math notation="LaTeX" id="Equation170"><![CDATA[
\begin{align}
\big(m_{n,\ell}^{C,S} \big)^2 = \frac{(n + \frac{1}{2})^2}{R_6^2}
+ \big( k \lambda_\ell^{C,S} \big)^2
\ge \left( \frac{1}{2 R_6} \right)^2\!.
\end{align}]]></tex-math></disp-formula></p>
<p>(iv) <inline-formula><tex-math notation="LaTeX" id="ImEquation1115"><![CDATA[$(P_0=P_1,P_2=P_3)=(-,+)$]]></tex-math></inline-formula>
<disp-formula id="ptx175-MA-24"><label>(A.24)</label><tex-math notation="LaTeX" id="Equation171"><![CDATA[
\begin{align}
&\Phi(x,z,v)= e^{\nu \sigma (y) } \left\{
\sum_{n=0}^{\infty} \sum_{\ell=1}^\infty
{\phi}_{n, \ell}^S(x) Z_{\nu} (\lambda_\ell^S z)g_{n+\frac{1}{2}}^C(v)\right. \cr
&\left.\hspace{3.7cm}
+\sum_{n=0}^{\infty} \sum_{\ell=0}^\infty
{\phi}_{n, \ell} ^C(x) Z_{\nu} (\lambda_\ell^C z) g_{n+\frac{1}{2}}^S(v) \right\}\!.
\end{align}]]></tex-math></disp-formula></p>
</sec>
<sec id="SECA.3"><title>A.3. 6D fermion field</title>
<p>Let us consider a free 6D Weyl fermion <inline-formula><tex-math notation="LaTeX" id="ImEquation1116"><![CDATA[$\Psi(x,y,v)$]]></tex-math></inline-formula> with <inline-formula><tex-math notation="LaTeX" id="ImEquation1117"><![CDATA[$\gamma_{6D}^7=+1$]]></tex-math></inline-formula>:
<disp-formula id="ptx175-MA-25"><label>(A.25)</label><tex-math notation="LaTeX" id="Equation172"><![CDATA[
\begin{align}
&\Psi(x,y,v)= e^{\frac{5}{2} \sigma (y)} \, \check \Psi , \check \Psi(x,z,v)=
\left[ \begin{matrix} \xi(x,z,v) \cr \eta(x,z,v) \cr 0 \cr 0 \end{matrix} \right] , \cr
&\gamma_{6D}^{7}= \begin{pmatrix} I_4& \cr &-I_4 \end{pmatrix}
=\gamma_{4D}^5\cdot (-i\gamma^5\gamma^6),
\gamma_{4D}^5= \begin{pmatrix} I_2&&& \cr &-I_2&& \cr &&I_2&\cr &&&-I_2 \end{pmatrix}.
\end{align}]]></tex-math></disp-formula></p>
<p>Its action is given by
<disp-formula id="ptx175-MA-26"><label>(A.26)</label><tex-math notation="LaTeX" id="Equation173"><![CDATA[
\begin{align}
&I=\int d^6x\sqrt{-\mbox{det}\,G} \,
\overline{\Psi}
\left\{\gamma^A E_A{}^{M}{\cal D}_M+ick\gamma^6\right\} \Psi \cr
&=\int d^4x \int_{0}^{2\pi R_6}dv \int_{1}^{z_L} \frac{dz}{k}
i [-\eta^\dagger,\xi^\dagger ]
\bigg[\begin{matrix} -kD_-(c)+i\partial_v &\sigma^\mu \partial_\mu \cr
\overline{\sigma}^\mu \partial_\mu & -kD_+(c)+i\partial_v \end{matrix} \bigg]
\bigg[\begin{matrix} \xi \cr \eta \end{matrix} \bigg] ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1118"><![CDATA[$D_\pm(c)$]]></tex-math></inline-formula> is defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-M4-5">4.5</xref>). As an example we consider the case in which the BCs are given by
<disp-formula id="ptx175-MA-27"><label>(A.27)</label><tex-math notation="LaTeX" id="Equation174"><![CDATA[
\begin{align}
\Psi(x,y_j-y,v_j-v) & =-i\gamma^5\gamma^6P_j\Psi(x,y_j+y,v_j+v) \cr
&=\gamma_{6D}^7\gamma_{4D}^5P_j\Psi(x,y_j+y,v_j+v) .
\end{align}]]></tex-math></disp-formula></p>
<p>The equations of motion are
<disp-formula id="ptx175-MA-28"><label>(A.28)</label><tex-math notation="LaTeX" id="Equation175"><![CDATA[
\begin{align}
&(-kD_- (c) + i\partial_v) \xi +\sigma^\mu\partial_\mu \eta=0 , \cr
&\overline{\sigma}^\mu\partial_\mu \xi +(-kD_+(c) +i\partial_v)\eta =0 .
\end{align}]]></tex-math></disp-formula></p>
<p>To find the eigenmodes, we separate 4D and the fifth- and sixth-dimensional parts as
<disp-formula id="ptx175-MA-29"><label>(A.29)</label><tex-math notation="LaTeX" id="Equation176"><![CDATA[
\begin{align}
&\xi(x,z,v)=\widetilde{\xi}(z,v)f_R(x) ,
\overline{\sigma}^\mu\partial_\mu f_R(x) =mf_L(x) , \cr
&\eta(x,z,v)=\widetilde{\eta}(z,v)f_L(x) ,
{\sigma}^\mu\partial_\mu f_L(x)=mf_R(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>It follows that
<disp-formula id="ptx175-MA-30"><label>(A.30)</label><tex-math notation="LaTeX" id="Equation177"><![CDATA[
\begin{align}
&(-kD_- (c) +i\partial_v) \widetilde{\xi} +m\widetilde{\eta}=0 , \cr
&m\widetilde{\xi}+ (-kD_+(c) +i\partial_v)\widetilde{\eta}=0 .
\end{align}]]></tex-math></disp-formula></p>
<p>In the <inline-formula><tex-math notation="LaTeX" id="ImEquation1119"><![CDATA[$y$]]></tex-math></inline-formula>-coordinate, <inline-formula><tex-math notation="LaTeX" id="ImEquation1120"><![CDATA[$\xi$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1121"><![CDATA[$\eta$]]></tex-math></inline-formula> satisfy the BCs
<disp-formula id="ptx175-MA-31"><label>(A.31)</label><tex-math notation="LaTeX" id="Equation178"><![CDATA[
\begin{align}
&
\begin{pmatrix} \xi \cr \eta \end{pmatrix} (x,y_j-y,v_j-v)=
P_j \begin{pmatrix} \xi \cr - \eta \end{pmatrix} (x,y_j+y,v_j+v) .
\end{align}]]></tex-math></disp-formula></p>
<p>(i) <inline-formula><tex-math notation="LaTeX" id="ImEquation1122"><![CDATA[$(P_0=P_1,P_2=P_3)=(+,+)$]]></tex-math></inline-formula></p>
<p>In this case,
<disp-formula id="ptx175-MA-32"><label>(A.32)</label><tex-math notation="LaTeX" id="Equation179"><![CDATA[
\begin{align}
&\xi(x,y_j-y,v_j-v)= + \xi(x,y_j+y,v_j+v) , \cr
&\eta(x,y_j-y,v_j-v)=-\eta(x,y_j+y,v_j+v) .
\end{align}]]></tex-math></disp-formula></p>
<p>The mode functions can be written as
<disp-formula id="ptx175-MA-33"><label>(A.33)</label><tex-math notation="LaTeX" id="Equation180"><![CDATA[
\begin{align}
\xi(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\xi}_n^C(y)f_n^C(v)
+\sum_{n=1}^{\infty}\widetilde{\xi}_n^S(y)f_n^S(v) \right\} f_R(x) , \cr
\eta(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\eta}_n^S(y)f_n^C(v)
+\sum_{n=1}^{\infty}\widetilde{\eta}_n^C(y)f_n^S(v) \right\} f_L(x) ,
\end{align}]]></tex-math></disp-formula>
where the <inline-formula><tex-math notation="LaTeX" id="ImEquation1123"><![CDATA[$f_n^C(v), f_n^S(v)$]]></tex-math></inline-formula> are defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-14">A.14</xref>). <inline-formula><tex-math notation="LaTeX" id="ImEquation1124"><![CDATA[$\widetilde{\xi}_n^{C,S}$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1125"><![CDATA[$\widetilde{\eta}_n^{C,S}$]]></tex-math></inline-formula> satisfy the BCs
<disp-formula id="ptx175-MA-34"><label>(A.34)</label><tex-math notation="LaTeX" id="Equation181"><![CDATA[
\begin{align}
&\widetilde{\xi}_n^C (y_j - y) = + \widetilde{\xi}_n^C (y_j + y) ,
\widetilde{\xi}_n^S (y_j - y) = - \widetilde{\xi}_n^S (y_j + y) , \cr
&\widetilde{\eta}_n^C (y_j - y) = + \widetilde{\eta}_n^C (y_j + y) ,
\widetilde{\eta}_n^S (y_j - y) = - \widetilde{\eta}_n^S (y_j + y) .
\end{align}]]></tex-math></disp-formula></p>
<p>By inserting Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-33">A.33</xref>) into Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-28">A.28</xref>), one finds equations for <inline-formula><tex-math notation="LaTeX" id="ImEquation1126"><![CDATA[$\widetilde \xi_n^{C,S}, \widetilde \eta_n^{C,S}$]]></tex-math></inline-formula> in the <inline-formula><tex-math notation="LaTeX" id="ImEquation1127"><![CDATA[$z$]]></tex-math></inline-formula>-coordinate (<inline-formula><tex-math notation="LaTeX" id="ImEquation1128"><![CDATA[$1 \le z \le z_L$]]></tex-math></inline-formula>):
<disp-formula id="ptx175-MA-35"><label>(A.35)</label><tex-math notation="LaTeX" id="Equation182"><![CDATA[
\begin{align}
&\begin{pmatrix} - k D_- & m \cr m & - kD_+ \end{pmatrix}
\begin{pmatrix} \widetilde{\xi}_0^C \cr \widetilde{\eta}_0^S \end{pmatrix} = 0 , \cr
&\begin{pmatrix} - k D_- & i \frac{n}{R_6} & m & 0 \cr
- i \frac{n}{R_6} & - k D_- &0 & m \cr
m & 0 & - k D_+ & i \frac{n}{R_6} \cr 0 & m & - i \frac{n}{R_6} & - k D_+ \end{pmatrix}
\begin{pmatrix} \widetilde{\xi}_n^C \cr \widetilde{\xi}_n^S \cr
\widetilde{\eta}_n^S \cr \widetilde{\eta}_n^C \end{pmatrix} = 0 \quad (n \ge 1).
\end{align}]]></tex-math></disp-formula></p>
<p>It follows that
<disp-formula id="ptx175-MA-36"><label>(A.36)</label><tex-math notation="LaTeX" id="Equation183"><![CDATA[
\begin{align}
&( k^2 D_+ D_- - m^2) \, \widetilde{\xi}_0^C =0 , \cr
&(k^2 D_- D_+ - m^2) \, \widetilde{\eta}_0^S = 0 , \cr
&\left( k^2D_+ D_- \mp\frac{2kcn}{R_6}\frac{1}{z}+\frac{n^2}{R_6^2}-m^2 \right)
\big(\widetilde{\xi}_n^C\pm i\widetilde{\xi}_n^S\big)=0 , \cr
&\left( k^2D_- D_+ \mp\frac{2kcn}{R_6}\frac{1}{z}+\frac{n^2}{R_6^2}-m^2 \right)
\big(\widetilde{\eta}_n^S \pm i \widetilde{\eta}_n^C\big)=0 \quad (n \ge 1) .
\end{align}]]></tex-math></disp-formula></p>
<p>The equations for the zero (<inline-formula><tex-math notation="LaTeX" id="ImEquation1129"><![CDATA[$n=0$]]></tex-math></inline-formula>) modes in the sixth dimension, <inline-formula><tex-math notation="LaTeX" id="ImEquation1130"><![CDATA[$\widetilde{\xi}_0^C$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1131"><![CDATA[$\widetilde{\eta}_0^S$]]></tex-math></inline-formula>, are reduced to the Bessel equation. In terms of the basis functions <inline-formula><tex-math notation="LaTeX" id="ImEquation1132"><![CDATA[$C_{R/L} (z; \lambda, c)$]]></tex-math></inline-formula>, <inline-formula><tex-math notation="LaTeX" id="ImEquation1133"><![CDATA[$S_{R/L} (z; \lambda, c)$]]></tex-math></inline-formula> given in Appendix B of Ref. [<xref ref-type="bibr" rid="B35">35</xref>], solutions satisfying the BCs are given by
<disp-formula id="ptx175-MA-37"><label>(A.37)</label><tex-math notation="LaTeX" id="Equation184"><![CDATA[
\begin{align}
&\widetilde{\xi}_{0, 0}^C = a_0 z^c , \widetilde{\eta}_{0, 0}^S = 0 ,
m_0 = 0 , \cr
&\widetilde{\xi}_{0, \ell}^C = a_\ell C_R(z; \lambda_\ell, c) ,
\widetilde{\eta}_{0, \ell}^S = a_\ell S_L(z; \lambda_\ell, c) , \cr
&S_L(1; \lambda_\ell, c) = 0 , m_\ell = k \lambda_\ell > 0 ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1134"><![CDATA[$a_\ell$]]></tex-math></inline-formula> is a normalization constant. There is one massless mode for the right-handed component. For the <inline-formula><tex-math notation="LaTeX" id="ImEquation1135"><![CDATA[$n \not= 0$]]></tex-math></inline-formula> modes the solutions are more involved.</p>
<p>(ii) <inline-formula><tex-math notation="LaTeX" id="ImEquation1136"><![CDATA[$(P_0=P_1,P_2=P_3)=(-,-)$]]></tex-math></inline-formula></p>
<p>In this case the mode functions of <inline-formula><tex-math notation="LaTeX" id="ImEquation1137"><![CDATA[$\xi$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1138"><![CDATA[$\eta$]]></tex-math></inline-formula> can be written as
<disp-formula id="ptx175-MA-38"><label>(A.38)</label><tex-math notation="LaTeX" id="Equation185"><![CDATA[
\begin{align}
\xi(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\xi}_n^S(y)f_n^C(v)
+\sum_{n=1}^{\infty}\widetilde{\xi}_n^C(y)f_n^S(v) \right\} f_R(x) , \cr
\eta(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\eta}_n^C(y)f_n^C(v)
+\sum_{n=1}^{\infty}\widetilde{\eta}_n^S(y)f_n^S(v) \right\} f_L(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>The equations for the zero (<inline-formula><tex-math notation="LaTeX" id="ImEquation1139"><![CDATA[$n=0$]]></tex-math></inline-formula>) modes in the sixth dimension, <inline-formula><tex-math notation="LaTeX" id="ImEquation1140"><![CDATA[$\widetilde{\xi}_0^S$]]></tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX" id="ImEquation1141"><![CDATA[$\widetilde{\eta}_0^C$]]></tex-math></inline-formula>, are
<disp-formula id="ptx175-MA-39"><label>(A.39)</label><tex-math notation="LaTeX" id="Equation186"><![CDATA[
\begin{align}
&\begin{pmatrix} - k D_- & m \cr m & - kD_+ \end{pmatrix}
\begin{pmatrix} \widetilde{\xi}_0^S \cr \widetilde{\eta}_0^C \end{pmatrix} = 0 ,
\end{align}]]></tex-math></disp-formula>
and solutions satisfying the BCs are given by
<disp-formula id="ptx175-MA-40"><label>(A.40)</label><tex-math notation="LaTeX" id="Equation187"><![CDATA[
\begin{align}
&\widetilde{\xi}_{0, 0}^S = 0 , \widetilde{\eta}_{0, 0}^C = a_0 z^{-c} ,
m_0 = 0 , \cr
&\widetilde{\xi}_{0, \ell}^S = a_\ell S_R(z; \lambda_\ell, c) ,
\widetilde{\eta}_{0, \ell}^C= a_\ell C_L(z; \lambda_\ell, c) , \cr
&S_R(1; \lambda_\ell, c) = 0 , m_\ell = k \lambda_\ell > 0 ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1142"><![CDATA[$a_\ell$]]></tex-math></inline-formula> is a normalization constant. There is one massless mode for the left-handed component.</p>
<p>(iii) <inline-formula><tex-math notation="LaTeX" id="ImEquation1143"><![CDATA[$(P_0=P_1,P_2=P_3)=(+,-)$]]></tex-math></inline-formula></p>
<p>In this case the mode functions can be written as
<disp-formula id="ptx175-MA-41"><label>(A.41)</label><tex-math notation="LaTeX" id="Equation188"><![CDATA[
\begin{align}
\xi(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\xi}_n^C(y)g_{n + \frac{1}{2}}^C(v)
+\sum_{n=0}^{\infty}\widetilde{\xi}_n^S(y)g_{n +\frac{1}{2}}^S(v) \right\} f_R(x) , \cr
\eta(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\eta}_n^S(y) g_{n +\frac{1}{2}}^C(v)
+\sum_{n=1}^{\infty}\widetilde{\eta}_n^C(y) g_{n +\frac{1}{2}}^S(v) \right\} f_L(x) ,
\end{align}]]></tex-math></disp-formula>
where <inline-formula><tex-math notation="LaTeX" id="ImEquation1144"><![CDATA[$g_{n +\frac{1}{2}}^{C,S} (v)$]]></tex-math></inline-formula> are defined in Eq. (<xref ref-type="disp-formula" rid="ptx175-MA-22">A.22</xref>). There are no zero modes in the sixth dimension.</p>
<p>(iv) <inline-formula><tex-math notation="LaTeX" id="ImEquation1145"><![CDATA[$(P_0=P_1,P_2=P_3)=(-,+)$]]></tex-math></inline-formula></p>
<p>In this case the mode functions can be written as
<disp-formula id="ptx175-MA-42"><label>(A.42)</label><tex-math notation="LaTeX" id="Equation189"><![CDATA[
\begin{align}
\xi(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\xi}_n^S(y)g_{n + \frac{1}{2}}^C(v)
+\sum_{n=0}^{\infty}\widetilde{\xi}_n^C(y)g_{n +\frac{1}{2}}^S(v) \right\} f_R(x) , \cr
\eta(x,y,v)& = \left\{
\sum_{n=0}^{\infty}\widetilde{\eta}_n^C(y) g_{n +\frac{1}{2}}^C(v)
+\sum_{n=1}^{\infty}\widetilde{\eta}_n^S(y) g_{n +\frac{1}{2}}^S(v) \right\} f_L(x) .
\end{align}]]></tex-math></disp-formula></p>
<p>There are no zero modes in the sixth dimension.</p>
</sec>
</sec>
<ref-list>
<title>References</title>
<ref id="B1"><label>[1]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>126</volume>, <fpage>309</fpage> (<year>1983</year>). (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0370-2693(83)90170-3">http://dx.doi.org/10.1016/0370-2693(83)90170-3</ext-link></comment>)</mixed-citation></ref>
<ref id="B2"><label>[2]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name></person-group> <source>Ann. Phys.</source> <volume>190</volume>, <fpage>233</fpage> (<year>1989</year>). (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0003-4916(89)90015-8">http://dx.doi.org/10.1016/0003-4916(89)90015-8</ext-link></comment>)</mixed-citation></ref>
<ref id="B3"><label>[3]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Davies</surname><given-names>A. T.</given-names></string-name> and <string-name name-style="western"><surname>McLachlan</surname><given-names>A.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>200</volume>, <fpage>305</fpage> (<year>1988</year>). (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0370-2693(88)90776-9">http://dx.doi.org/10.1016/0370-2693(88)90776-9</ext-link></comment>)</mixed-citation></ref>
<ref id="B4"><label>[4]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Davies</surname><given-names>A. T.</given-names></string-name> and <string-name name-style="western"><surname>McLachlan</surname><given-names>A.</given-names></string-name></person-group> <source>Nucl. Phys. B</source> <volume>317</volume>, <fpage>237</fpage> (<year>1989</year>). (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0550-3213(89)90569-5">http://dx.doi.org/10.1016/0550-3213(89)90569-5</ext-link></comment>)</mixed-citation></ref>
<ref id="B5"><label>[5]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> <string-name name-style="western"><surname>Inami</surname><given-names>T.</given-names></string-name> and <string-name name-style="western"><surname>Lim</surname><given-names>C. S.</given-names></string-name></person-group> <source>Mod. Phys. Lett. A</source> <volume>13</volume>, <fpage>2601</fpage> (<year>1998</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-th/9805067">arXiv:hep-th/9805067</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-th/9805067">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1142/S021773239800276X">http://dx.doi.org/10.1142/S021773239800276X</ext-link></comment>)</mixed-citation></ref>
<ref id="B6"><label>[6]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Agashe</surname><given-names>K.</given-names></string-name> <string-name name-style="western"><surname>Contino</surname><given-names>R.</given-names></string-name> and <string-name name-style="western"><surname>Pomarol</surname><given-names>A.</given-names></string-name></person-group> <source>Nucl. Phys. B</source> <volume>719</volume>, <fpage>165</fpage> (<year>2005</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0412089">arXiv:hep-ph/0412089</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0412089">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.nuclphysb.2005.04.035">http://dx.doi.org/10.1016/j.nuclphysb.2005.04.035</ext-link></comment>)</mixed-citation></ref>
<ref id="B7"><label>[7]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Medina</surname><given-names>A. D.</given-names></string-name> <string-name name-style="western"><surname>Shah</surname><given-names>N. R.</given-names></string-name> and <string-name name-style="western"><surname>Wagner</surname><given-names>C. E. M.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>76</volume>, <fpage>095010</fpage> (<year>2007</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/0706.1281">arXiv:0706.1281</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+0706.1281">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.76.095010">http://dx.doi.org/10.1103/PhysRevD.76.095010</ext-link></comment>)</mixed-citation></ref>
<ref id="B8"><label>[8]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Sakamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Prog. Theor. Phys.</source> <volume>118</volume>, <fpage>935</fpage> (<year>2007</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0703212">arXiv:hep-ph/0703212</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0703212">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1143/PTP.118.935">http://dx.doi.org/10.1143/PTP.118.935</ext-link></comment>)</mixed-citation></ref>
<ref id="B9"><label>[9]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> <string-name name-style="western"><surname>Oda</surname><given-names>K.</given-names></string-name> <string-name name-style="western"><surname>Ohnuma</surname><given-names>T.</given-names></string-name> and <string-name name-style="western"><surname>Sakamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>78</volume>, <fpage>096002</fpage> (<year>2008</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/0806.0480">arXiv:0806.0480</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+0806.0480">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1103/PhysRevD.78.096002">https://doi.org/10.1103/PhysRevD.78.096002</ext-link></comment>)</mixed-citation></ref>
<ref id="B10"><label>[10]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> <string-name name-style="western"><surname>Noda</surname><given-names>S.</given-names></string-name> and <string-name name-style="western"><surname>Uekusa</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Phys.</source> <volume>123</volume>, <fpage>757</fpage> (<year>2010</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/0912.1173">arXiv:0912.1173</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+0912.1173">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1143/PTP.123.757">http://dx.doi.org/10.1143/PTP.123.757</ext-link></comment>)</mixed-citation></ref>
<ref id="B11"><label>[11]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Funatsu</surname><given-names>S.</given-names></string-name> <string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> <string-name name-style="western"><surname>Orikasa</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Shimotani</surname><given-names>T.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>722</volume>, <fpage>94</fpage> (<year>2013</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1301.1744">arXiv:1301.1744</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1301.1744">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.physletb.2013.03.040">http://dx.doi.org/10.1016/j.physletb.2013.03.040</ext-link></comment>)</mixed-citation></ref>
<ref id="B12"><label>[12]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Funatsu</surname><given-names>S.</given-names></string-name> <string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> <string-name name-style="western"><surname>Orikasa</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Shimotani</surname><given-names>T.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>89</volume>, <fpage>095019</fpage> (<year>2014</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1404.2748">arXiv:1404.2748</ext-link> [hep-ph] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1404.2748">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.89.095019">http://dx.doi.org/10.1103/PhysRevD.89.095019</ext-link></comment>)</mixed-citation></ref>
<ref id="B13"><label>[13]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Funatsu</surname><given-names>S.</given-names></string-name> <string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> <string-name name-style="western"><surname>Orikasa</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Shimotani</surname><given-names>T.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <volume>2014</volume>, <fpage>113B01</fpage> (<year>2014</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1407.3574">arXiv:1407.3574</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1407.3574">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptu146">http://dx.doi.org/10.1093/ptep/ptu146</ext-link></comment>)</mixed-citation></ref>
<ref id="B14"><label>[14]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Funatsu</surname><given-names>S.</given-names></string-name> <string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> and <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>92</volume>, <fpage>115003</fpage> (<year>2015</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1510.06550">arXiv:1510.06550</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1510.06550">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.92.115003">http://dx.doi.org/10.1103/PhysRevD.92.115003</ext-link></comment>)</mixed-citation></ref>
<ref id="B15"><label>[15]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Funatsu</surname><given-names>S.</given-names></string-name> <string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Orikasa</surname><given-names>Y.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>95</volume>, <fpage>035032</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1612.03378">arXiv:1612.03378</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1612.03378">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.95.035032">http://dx.doi.org/10.1103/PhysRevD.95.035032</ext-link></comment>)</mixed-citation></ref>
<ref id="B16"><label>[16]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Funatsu</surname><given-names>S.</given-names></string-name> <string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Orikasa</surname><given-names>Y.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>775</volume>, <fpage>297</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1705.05282">arXiv:1705.05282</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1705.05282">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.physletb.2017.10.068">http://dx.doi.org/10.1016/j.physletb.2017.10.068</ext-link></comment>)</mixed-citation></ref>
<ref id="B17"><label>[17]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Kawamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Prog. Theor. Phys.</source> <volume>103</volume>, <fpage>613</fpage> (<year>2000</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/9902423">arXiv:hep-ph/9902423</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/9902423">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1143/PTP.103.613">http://dx.doi.org/10.1143/PTP.103.613</ext-link></comment>)</mixed-citation></ref>
<ref id="B18"><label>[18]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Kawamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Prog. Theor. Phys.</source> <volume>105</volume>, <fpage>691</fpage> (<year>2001</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0012352">arXiv:hep-ph/0012352</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0012352">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1143/PTP.105.691">http://dx.doi.org/10.1143/PTP.105.691</ext-link></comment>)</mixed-citation></ref>
<ref id="B19"><label>[19]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Kawamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Prog. Theor. Phys.</source> <volume>105</volume>, <fpage>999</fpage> (<year>2001</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0012125">arXiv:hep-ph/0012125</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0012125">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1143/PTP.105.999">http://dx.doi.org/10.1143/PTP.105.999</ext-link></comment>)</mixed-citation></ref>
<ref id="B20"><label>[20]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Frigerio</surname><given-names>M.</given-names></string-name> <string-name name-style="western"><surname>Serra</surname><given-names>J.</given-names></string-name> and <string-name name-style="western"><surname>Varagnolo</surname><given-names>A.</given-names></string-name></person-group> <source>J. High Energy Phys.</source> <volume>06</volume>, <fpage>029</fpage> (<year>2011</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1103.2997">arXiv:1103.2997</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1103.2997">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/JHEP06(2011)029">http://dx.doi.org/10.1007/JHEP06(2011)029</ext-link></comment>)</mixed-citation></ref>
<ref id="B21"><label>[21]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Yamamoto</surname><given-names>K.</given-names></string-name></person-group> <source>Nucl. Phys. B</source> <volume>883</volume>, <fpage>45</fpage> (<year>2014</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1401.0466">arXiv:1401.0466</ext-link> [hep-th]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1401.0466">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.nuclphysb.2014.03.017">http://dx.doi.org/10.1016/j.nuclphysb.2014.03.017</ext-link></comment>)</mixed-citation></ref>
<ref id="B22"><label>[22]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <volume>2017</volume>, <fpage>061B01</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1704.08827">arXiv:1704.08827</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1704.08827">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptx088">http://dx.doi.org/10.1093/ptep/ptx088</ext-link></comment>)</mixed-citation></ref>
<ref id="B23"><label>[23]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <volume>2017</volume>, <fpage>101B01</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1708.02078">arXiv:1708.02078</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1708.02078">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptx135">http://dx.doi.org/10.1093/ptep/ptx135</ext-link></comment>)</mixed-citation></ref>
<ref id="B24"><label>[24]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Burdman</surname><given-names>G.</given-names></string-name> and <string-name name-style="western"><surname>Nomura</surname><given-names>Y.</given-names></string-name></person-group> <source>Nucl. Phys. B</source> <volume>656</volume>, <fpage>3</fpage> (<year>2003</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0210257">arXiv:hep-ph/0210257</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0210257">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0550-3213(03)00088-9">http://dx.doi.org/10.1016/S0550-3213(03)00088-9</ext-link></comment>)</mixed-citation></ref>
<ref id="B25"><label>[25]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Haba</surname><given-names>N.</given-names></string-name> <string-name name-style="western"><surname>Harada</surname><given-names>M.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Kawamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Nucl. Phys. B</source> <volume>657</volume>, <fpage>169</fpage> (<year>2003</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0212035">arXiv:hep-ph/0212035</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0212035">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/S0550-3213(03)00142-1">http://dx.doi.org/10.1016/S0550-3213(03)00142-1</ext-link></comment>)</mixed-citation></ref>
<ref id="B26"><label>[26]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Haba</surname><given-names>N.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Kawamura</surname><given-names>Y.</given-names></string-name></person-group> <source>Prog. Theor. Phys.</source> <volume>111</volume>, <fpage>265</fpage> (<year>2004</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0309088">arXiv:hep-ph/0309088</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0309088">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1143/PTP.111.265">http://dx.doi.org/10.1143/PTP.111.265</ext-link></comment>)</mixed-citation></ref>
<ref id="B27"><label>[27]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Haba</surname><given-names>N.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> <string-name name-style="western"><surname>Kawamura</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Yamashita</surname><given-names>T.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>70</volume>, <fpage>015010</fpage> (<year>2004</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0401183">arXiv:hep-ph/0401183</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0401183">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.70.015010">http://dx.doi.org/10.1103/PhysRevD.70.015010</ext-link></comment>)</mixed-citation></ref>
<ref id="B28"><label>[28]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Lim</surname><given-names>C. S.</given-names></string-name> and <string-name name-style="western"><surname>Maru</surname><given-names>N.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>653</volume>, <fpage>320</fpage> (<year>2007</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/0706.1397">arXiv:0706.1397</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+0706.1397">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.physletb.2007.07.053">http://dx.doi.org/10.1016/j.physletb.2007.07.053</ext-link></comment>)</mixed-citation></ref>
<ref id="B29"><label>[29]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Kojima</surname><given-names>K.</given-names></string-name> <string-name name-style="western"><surname>Takenaga</surname><given-names>K.</given-names></string-name> and <string-name name-style="western"><surname>Yamashita</surname><given-names>T.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>84</volume>, <fpage>051701(R)</fpage> (<year>2011</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1103.1234">arXiv:1103.1234</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1103.1234">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.84.051701">http://dx.doi.org/10.1103/PhysRevD.84.051701</ext-link></comment>)</mixed-citation></ref>
<ref id="B30"><label>[30]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Kojima</surname><given-names>K.</given-names></string-name> <string-name name-style="western"><surname>Takenaga</surname><given-names>K.</given-names></string-name> and <string-name name-style="western"><surname>Yamashita</surname><given-names>T.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>95</volume>, <fpage>015021</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1608.05496">arXiv:1608.05496</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1608.05496">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.95.015021">http://dx.doi.org/10.1103/PhysRevD.95.015021</ext-link></comment>)</mixed-citation></ref>
<ref id="B31"><label>[31]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Kojima</surname><given-names>K.</given-names></string-name> <string-name name-style="western"><surname>Takenaga</surname><given-names>K.</given-names></string-name> and <string-name name-style="western"><surname>Yamashita</surname><given-names>T.</given-names></string-name></person-group> <source>J. High Energy Phys.</source> <volume>06</volume>, <fpage>018</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1704.04840">arXiv:1704.04840</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1704.04840">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/JHEP06(2017)018">http://dx.doi.org/10.1007/JHEP06(2017)018</ext-link></comment>)</mixed-citation></ref>
<ref id="B32"><label>[32]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <volume>2015</volume>, <fpage>111B01</fpage> (<year>2015</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1504.03817">arXiv:1504.03817</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1504.03817">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptv153">http://dx.doi.org/10.1093/ptep/ptv153</ext-link></comment>)</mixed-citation></ref>
<ref id="B33"><label>[33]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>PoS PLANCK</source> <volume>2015</volume>, <fpage>058</fpage> (<year>2015</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1511.01674">arXiv:1511.01674</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1511.01674">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="https://pos.sissa.it/258/058/pdf">https://pos.sissa.it/258/058/pdf</ext-link></comment>)</mixed-citation></ref>
<ref id="B34"><label>[34]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <volume>2016</volume>, <fpage>043B02</fpage> (<year>2016</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1512.05559">arXiv:1512.05559</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1512.05559">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptw023">http://dx.doi.org/10.1093/ptep/ptw023</ext-link></comment>)</mixed-citation></ref>
<ref id="B35"><label>[35]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Furui</surname><given-names>A.</given-names></string-name> <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <fpage>2016</fpage>, <fpage>093B01</fpage> (<year>2016</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1606.07222">arXiv:1606.07222</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1606.07222">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptw116">http://dx.doi.org/10.1093/ptep/ptw116</ext-link></comment>)</mixed-citation></ref>
<ref id="B36"><label>[36]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name></person-group> <source>Int. J. Mod. Phys. A</source> <volume>31</volume>, <fpage>1630031</fpage> (<year>2016</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1606.08108">arXiv:1606.08108</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1606.08108">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1142/S0217751X16300313">https://doi.org/10.1142/S0217751X16300313</ext-link></comment>)</mixed-citation></ref>
<ref id="B37"><label>[37]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name></person-group> [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1702.08161">arXiv:1702.08161</ext-link> [hep-ph] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1702.08161">Search <sc>in</sc>SPIRE</ext-link>].</mixed-citation></ref>
<ref id="B38"><label>[38]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name> and <string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> <source>Prog. Theor. Exp. Phys.</source> <volume>2017</volume>, <fpage>091B01</fpage> (<year>2017</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1706.03503">arXiv:1706.03503</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1706.03503">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1093/ptep/ptx124">http://dx.doi.org/10.1093/ptep/ptx124</ext-link></comment>)</mixed-citation></ref>
<ref id="B39"><label>[39]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Mirabelli</surname><given-names>E. A.</given-names></string-name> and <string-name name-style="western"><surname>Peskin</surname><given-names>M. E.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>58</volume>, <fpage>065002</fpage> (<year>1998</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-th/9712214">arXiv:hep-th/9712214</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-th/9712214">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.58.065002">http://dx.doi.org/10.1103/PhysRevD.58.065002</ext-link></comment>)</mixed-citation></ref>
<ref id="B40"><label>[40]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Minkowski</surname><given-names>P.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>67</volume>, <fpage>421</fpage> (<year>1977</year>). (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0370-2693(77)90435-X">http://dx.doi.org/10.1016/0370-2693(77)90435-X</ext-link></comment>)</mixed-citation></ref>
<ref id="B41"><label>[41]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Randall</surname><given-names>L.</given-names></string-name> and <string-name name-style="western"><surname>Sundrum</surname><given-names>R.</given-names></string-name></person-group> <source>Phys. Rev. Lett.</source> <volume>83</volume>, <fpage>3370</fpage> (<year>1999</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/9905221">arXiv:hep-ph/9905221</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/9905221">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevLett.83.3370">http://dx.doi.org/10.1103/PhysRevLett.83.3370</ext-link></comment>)</mixed-citation></ref>
<ref id="B42"><label>[42]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name name-style="western"><surname>McKay</surname><given-names>W. G.</given-names></string-name> and <string-name name-style="western"><surname>Patera</surname><given-names>J.</given-names></string-name></person-group> <article-title>Tables of Dimensions, Indices, and Branching Rules for Representations of Simple Lie Algebras</article-title> (<publisher-name>Marcel Dekker, Inc.,</publisher-name> <publisher-loc>New York</publisher-loc>, <year>1981</year>).</mixed-citation></ref>
<ref id="B43"><label>[43]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Slansky</surname><given-names>R.</given-names></string-name></person-group> <source>Phys. Rept.</source> <volume>79</volume>, <fpage>1</fpage> (<year>1981</year>). (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/0370-1573(81)90092-2">http://dx.doi.org/10.1016/0370-1573(81)90092-2</ext-link></comment>)</mixed-citation></ref>
<ref id="B44"><label>[44]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Yamatsu</surname><given-names>N.</given-names></string-name></person-group> [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1511.08771">arXiv:1511.08771</ext-link> [hep-ph] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1511.08771">Search <sc>in</sc>SPIRE</ext-link>].</mixed-citation></ref>
<ref id="B45"><label>[45]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Falkowski</surname><given-names>A.</given-names></string-name></person-group> <source>Phys. Rev. D</source> <volume>75</volume>, <fpage>025017</fpage> (<year>2007</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/hep-ph/0610336">arXiv:hep-ph/0610336</ext-link>] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+hep-ph/0610336">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1103/PhysRevD.75.025017">http://dx.doi.org/10.1103/PhysRevD.75.025017</ext-link></comment>)</mixed-citation></ref>
<ref id="B46"><label>[46]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name name-style="western"><surname>Hatanaka</surname><given-names>H.</given-names></string-name> and <string-name name-style="western"><surname>Hosotani</surname><given-names>Y.</given-names></string-name></person-group> <source>Phys. Lett. B</source> <volume>713</volume>, <fpage>481</fpage> (<year>2012</year>) [<ext-link ext-link-type="uri" xlink:href="http://www.arxiv.org/abs/1111.3756">arXiv:1111.3756</ext-link> [hep-ph]] [<ext-link ext-link-type="uri" xlink:href="http://www.inspirehep.net/search?p=find+EPRINT+1111.3756">Search <sc>in</sc>SPIRE</ext-link>]. (<comment><ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.1016/j.physletb.2012.06.039">http://dx.doi.org/10.1016/j.physletb.2012.06.039</ext-link></comment>)</mixed-citation></ref>
</ref-list>
</back>
</article>