Hyperparameters. Using the Fourier neural operator implementation of [6], for which a well-explained documentation can be found on GitHub, we set up a 1D tensorized Fourier neural operator (TFNO) implemented in pytorch with the following hyperparameters: number of Fourier modes :n_modes=50,number of hidden channels :hidden_channels=64,number of projection channels :projection_channels=512,number of layers :n_layers=4,type of factorization :factorization=“tucker,”rank :rank=0.01.This is a model with 76,849 parameters that are tuned during the training to produce an optimal NO. The training optimization was performed using ADAM [36] with learning rate 10-3, weight decay 10-5 and batch size 256. Varying the above hyperparameters did not result in significant variations of the results.

Training. The training dataset is prepared in the following manner. We generate random samples of amplitudes with finite partial wave expansions F(z)=B(z)eiϕ(z)=∑ℓ=0L(2ℓ+1)sinδℓeiδℓPℓ(z)(11)sampling the random phase shifts δℓ from a uniform distribution. 100,000 samples are collected for L=1,2, and 3, separately, providing a total of 300,000 amplitudes. For each of these amplitudes we read off their modulus B(z) and the sine of their phase sinϕ(z). Afterward, we discretize z=cosθ∈[-1,1] on a uniform grid of 100 points⁵

A remarkable feature of NOs is their capacity to efficiently implement zero-shot superresolution [37]. In the context of quantum 2→2 scattering, this gives us the ability to train on a grid of, say, 100 points and then easily make accurate predictions at higher resolutions. We did not see the need to go beyond the 100-point resolution in this problem, but it is good to keep in mind that this possibility exists.

We emphasize that once the trained NO has been obtained, it can be used to make very quick predictions for any input modulus B(z), in stark contrast with the PINN approach, where retraining from scratch is needed for every new input.

Once the NO has trained on known samples, we investigate how well it generalizes both on the same class of data (training/test dataset), as well as on different classes of never-before-seen data. Predicting the phases of amplitudes in the latter case would indicate that the NO is able to learn the unitarity relation (5) and effectively solve it without having direct access to it.

Tests within the training-test dataset. For starters, we can ask about the quality of predictions inside the training/test dataset. In Fig. 1 we plot the ground truth (blue) and predictions (orange) of the trained NO for sinϕ(z) of three randomly chosen samples from the test dataset with L=1,2,3, respectively. The plots of the ground truth and prediction are visually indistinguishable, indicating that the NO has trained well. To get a sense of the numerical size of the error in the plots of Fig. 1, for L=1 the average relative error between the ground truth and prediction across the whole z grid is 0.4%. For L=2 it is 1.1% and for L=3 it is 0.9%. These numbers are typical in the test dataset. The percentage of samples that exhibit average relative error above 10% is 5.2%.

A first sample of tests on moduli with infinite partial wave expansions. A more interesting question concerns the extent to which the NO can generalize outside the training dataset. The first case we would like to discuss here are amplitudes with an infinite partial wave expansion. For concreteness, we will consider two examples of linear and quadratic moduli that were analyzed also in Ref. [12]. Later we will scan more extensively over the predictions for amplitudes with linear, quadratic, as well as cubic moduli.

The first example concerns amplitudes with linear modulus B(z)=az+b (and b>|a| for positivity). It is straightforward to check that these amplitudes do not have a finite partial wave expansion.⁶

For a detailed discussion see [12].

The second example refers to the quadratic modulus B(z)=12(z2+1), which was also discussed in Ref. [12]. This amplitude has Martin parameter sinμ=0.867 and can once again be determined numerically by solving the unitarity equation (5) with a simple iteration scheme. In the bottom left plot of Fig. 2 we present in orange and blue, respectively, the sinϕ for the NO prediction and the solution of the unitarity equation. Once again, the two plots are visibly close. In the bottom right plot of Fig. 2 we also present the relative difference, which is now of the order of O(10-2) for most points. It increases near z=-1, where the prediction in the depicted run was less accurate.

Scans on linear, quadratic and cubic moduli. We can test the quality of the NO predictions on moduli with infinite partial wave expansions more extensively, by performing a scan over a wide grid of linear, quadratic and cubic moduli. To quantify the quality of the predictions we compute the loss in the unitarity condition (5), L≔1Nc∑zi(sinϕ(zi)-14πB(zi)∫-11dz1∫02πdϕ1B(z1)⁢B(z2i)cos(ϕ(z1)-ϕ(z2i)))2,(13)where z2i is computed as in (6) with z=zi. The sum is over the points zi of the collocation z grid and the average is obtained by dividing with the number Nc of collocation points. In our runs Nc=100. The integrals in (13) were computed numerically using the trapezoidal rule.⁷

The results presented in this section used the fixed grid of 100 collocation points in the NO training in order to apply the trapezoidal rule. It is straightforward to achieve higher numerical accuracy in the numerical computation of the integrals in (13) using higher resolution grids with NO zero-shot superresolution.

For the linear moduli B(z)=az+b we considered a grid of 180×150 points on the (a,b) plane for a∈[-0.5,2], b∈[0,2] and b>|a|. The heat map of the log10L values on this grid appears on the top left plot of Fig. 3. The corresponding heat map in Ref. [12] appears in Fig. 3 of that paper. Reference [12] computed on a grid of 75×60 points in the (a,b) plane, retraining the neural networks for 2K epochs to obtain each point. Instead, we are evaluating the already trained NO at each point producing a heat map on a finer grid within approximately 20 sec.

In the top left plot of Fig. 3 we observe a distinct, blue-colored, low-loss region inside the sinμ=1 contour, precisely like the one detected by PINNs for linear moduli in [12]. The main difference with the PINN result is that its lowest log10 losses are in the vicinity of -8, whereas our corresponding values are in the vicinity of -5. That amounts to a difference in loss between the two methods at the level of 3 orders of magnitude. This is expected, since the PINN performs a dedicated optimization search for each input modulus B(z) explicitly using the unitarity condition (5), whereas the NO trains on a completely different class of inputs to produce a prediction outside its training dataset without using the unitarity condition. In that sense, the NO results in Fig. 3 are impressive and provide a distinct indication that the NO has been able to generalize well within the infinite partial wave amplitudes with linear moduli. Of course, by simply looking at the heat map of Fig. 3 one cannot really deduce where one should put the cutoff that separates the predictions that are consistent with unitarity from the ones that are inconsistent with unitarity. The same issue also exists within the PINN approach, but there it is slightly mitigated by the lower losses of the corresponding results. We will have to say more about how to address this difficulty in the next subsection.

For the quadratic moduli B(z)=cz2+d we also considered a grid of 180×150 points on the (c,d) plane for c∈[-0.5,,5.5], d∈[0,1.5] and c>|d|. The log10L heat map on this grid is depicted on the top right plot of Fig. 3. The corresponding heat map from Ref. [12] appears in Fig. 5 of that paper. Once again, we observe the formation of a low-loss region inside the sinμ=1 contour, which is comparable with the result in Fig. 5 of Ref. [12], suggesting that the NO has been able to generalize to this class of amplitudes as well. Similar to the linear case, the NO losses are higher by roughly 3 orders of magnitude compared to the PINN losses of [12].

Finally, in the bottom plot of Fig. 3 we present the log10 unitarity loss for cubic moduli of the form B(z)=cz3+d. Such amplitudes were not discussed in Ref. [12]. The resulting heat map is comparable to the linear-moduli heat map, exhibiting a distinct low-loss region inside the sinμ=1 contour, as expected.

To summarize, in all three cases of infinite partial wave amplitudes analyzed in this subsection, the picture that emerges is impressively consistent with expectations from the analysis of the unitarity equation (5), suggesting that the NO has learned nontrivial features of that equation without having access to it. The results also exhibit some of the weaknesses of the approach: (a)

The lowest losses are a few orders of magnitude higher than those produced by PINNs. That makes it harder to detect, without prior knowledge, the boundary between valid predictions consistent with unitarity and invalid predictions inconsistent with unitarity.

Tests on higher-L finite partial wave amplitudes. Another class of amplitudes outside the training-test dataset are the finite partial wave amplitudes with values of L>3. Exploring the quality of the NO predictions in this class shows that already at L=4 the NO fails to make any accurate predictions. This is, for example, apparent in the predictions presented at the bottom plot of Fig. 6, which depicts as a thick gray curve the exact result and as blue and orange dots the predictions of two separately trained NOs. The corresponding input modulus B(z) appears on the top left plot of Fig. 6.

This case demonstrates that with the above-mentioned training on L=1,2,3 amplitudes the NO cannot fully reconstruct the unitarity equation, which would allow for valid predictions with arbitrary input modulus B(z). It has been able to learn nontrivial elements of the unitarity constraints, but not all the information that these constraints entail.

Hyperparameters. The results presented in this section were obtained with a 1D TFNO that has the same neural network and optimization hyperparameters as the model in Sec. IV A 1. However, in this case a different tensorization approach yields a larger model with a total number of 874,241 parameters (an order of magnitude larger than the one in the previous model of Sec. IV A 1).

Training. We are using the same z grid as in Sec. IV A 1 with 100 collocation points. The output vector of the NO is therefore 101 dimensional, including an extra element v101 characterizing the validity of the output. In our runs we chose to train by assigning the value 10 to valid input-output pairs and -10 to false pairs. The fidelity index was defined as F≔10+v10120, which assigns 1 to valid pairs and 0 to invalid ones.

We explored the results of training for a variety of datasets with varying fractions of true and false inputs/outputs. As one would expect, we observed that the quality of the classification output decreased when the fraction of false pairs was reduced. Here, we report results for a training-test dataset of 400,000 samples with the following composition: 75,000 true pairs for each of the L=1,2,3 amplitudes and 175,000 false pairs. This yields a 43.75% fraction of false samples. The inputs and outputs of the false samples were generated randomly from two different groups of L=3 amplitudes. We reserved 1200 samples for testing and the samples were randomly mixed to put the true and false pairs in random order. With these specifications, we trained 56 independent NOs for 1500 epochs.

Tests within the training-test dataset. The accuracy of the fidelity-index prediction can be probed by computing the difference ΔF(sample)≔|Fpred(sample)-Fground truth(sample)|(14)between the predicted fidelity index and its ground truth, for each sample in the test dataset of 1200 samples and separately for each of the above 56 trained NOs. Assuming that a prediction is considered correct when ΔF<C(15)for some arbitrarily chosen C<1, we can go through the samples and register the number of times the inequality (15) is satisfied. This produces a success ratio Si(C) for the ith NO. We can further average this success ratio over the NOs; we call the corresponding quantity S¯F(C). For C=0.2 and 0.3 we find S¯F(0.2)≃73.94%,S¯F(0.3)≃75.34%.(16)The values SF(i)(0.2) and SF(i)(0.3) for the individual NOs are very close to the above averages. In other words, there is little variation between the different NOs in this datum. This suggests that (with the above cutoffs for ΔF) the fidelity index makes the right classification roughly 75% of the time, which is an encouraging sign of classification capacity but not an impressively high percentage.

We can also rephrase the above test in terms of a mean fidelity index F¯(sample), which is defined for each sample as the average over the corresponding fidelity indices of all trained NOs. We can then define the difference ΔF¯(sample) as in (14) by replacing F with F¯, placing a cutoff as in (15) and computing the average S¯F¯(C) over the samples. For this quantity we find S¯F¯(0.2)≃67.42%,S¯F¯(0.3)≃73%,(17)which is comparable to the previous result. We conclude that the average of the fidelity index over the NOs did not improve the classification capacity in this context.

These observations provide useful information about the performance of the NO as a classifier, but do not tell the whole story. In particular, we would now like to argue that the above tests do not really address some important aspects of the classification performance. Indeed, when one uses a NO to make a prediction for a never-before-seen modulus, it is very useful to know whether a predicted phase truly exists and can be considered correct with confidence, given an appropriately high fidelity index. Everything outside a small range of high fidelity values near 1 can be considered either as plausibly false for an existing phase or false because a phase does not exist. This viewpoint rephrases the way we should measure the success ratio in the test dataset.

Accordingly, we can now perform the following test. For each individually trained NO, we scan through the test dataset and count how many times the NO falsely affirms that the prediction is correct. The criterion for a prediction to be declared correct is |Fpred(sample)-1|<C.(18)As we scan through the samples we count the cases where this inequality is satisfied and the ground-truth fidelity index vanishes (namely, the sample is false). That gives a percentage⁸

We define the ratio that gives this percentage as the number of false predictions satisfying (18) divided by the total number of predictions satisfying (18).

We notice that the predictions of a true solution are only 4.08% of the times wrong when the fidelity index is inside the interval [0.99, 1.01]. This implies relatively high confidence in such predictions. We also notice that this criterion captures 66.5% of the total number of true samples in the test dataset. As we increase C (and with it the corresponding range of accepted fidelity indices) the fraction of wrong predictions increases and our confidence goes down, but the fraction of correct true predictions saturates. This implies that a small value of C at the level of 0.01 is a preferable choice.

It is also interesting to reevaluate these numbers using the mean fidelity index F¯. In that case, we are first averaging the fidelity index over the trained NOs for a given sample to produce the corresponding mean fidelity index F¯pred(sample), then we use it to impose a criterion like (18) and accordingly count which of the allowed samples are false predictions. This procedure yields a percentage of failure fF¯(C) for the “mean NO” and the analog of (20) is fF¯(0.01)=1.42%,correct true predictionstotal no.of true samples=61.6%fF¯(0.02)=1.56%,correct true predictionstotal no. of true samples=65.2%fF¯(0.05)=1.97%,correct true predictionstotal no. of true samples=66.4%fF¯(0.10)=2.58%,correct true predictionstotal no. of true samples=67.0%fF¯(0.20)=4.63%,correct true predictionstotal no. of true samples=67.1%.(21)

We observe that the mean fidelity index produces a lower percentage of failure at the same value of C (compared to the index of individual NOs) and, therefore, can be used to make predictions of correct phases with greater confidence. For example, the percentage of failed true predictions for the mean fidelity index at C=0.01 is only 1.42%, compared to 4.08% of the individual fidelity indices. The fraction of correct true predictions is comparable in both cases meaning that the mean fidelity index continues to detect essentially the same number of true samples with higher confidence.

Correlations between the unitarity loss and the fidelity index. As a further calibrating question we can ask whether the fidelity index correlates with the values of the unitarity loss. As an example, in Fig. 4 we contrast the heat map of the log10 unitarity loss and the heat map of the fidelity index for the predictions of one of the 56 trained NOs on the quadratic moduli B(z)=cz2+d. There is visible correlation between the plots, and this is typical in many training runs both for linear and quadratic moduli. It is difficult, however, to make a precise quantitative statement about their relation.

We also notice a clean separation between the predictions with high fidelity index (above 0.99) and predictions with low fidelity index (below 0.95). This is an interesting feature that correlates well with the above-mentioned observations about the fidelity index and its success rate. Unlike the unitarity loss, which varies smoothly between true and false predictions, the fidelity index appears to provide a more sharp acceptance/rejection criterion.

The results of the 56 trained NOs warrant some additional observations. First, we notice that the presence of the extra label that classifies the sample as true or false has affected the nature of the predictions across the landscape of input moduli. This is visible in the comparison of the unitarity losses in the top right heat map of Fig. 3 against the heat map on the left of Fig. 4.

Second, the unitarity losses for the predictions of the 56 NOs that included the fidelity label are typically slightly higher than those for the predictions of the NOs in Sec. IV A 2, which did not involve any training on false samples. This is expected, since we were training with 300,000 true samples in Sec. IV A 2, whereas the training here involves a smaller number of true samples, 225,000.

Third, across the set of the 56 different NOs, we observed significant variation in the heat maps of the predicted phases and their fidelity index. This observation hints at the complexities of the training process in this context and makes it harder to extract invariant information from individual training runs. It is therefore interesting to explore whether we can obtain information independent of the fluctuations of individual training iterations, reflecting real properties of the system, by collecting statistics from multiple runs.

Performance of the mean fidelity index. The above discussion is further motivation in favor of the use of the mean fidelity index F¯, which involves an average over independent NOs. In what follows, we evaluate F¯ across the 56 previously trained NOs and plot the results in Fig. 5 across the landscape of linear and quadratic moduli. We observed that as we incorporated more and more NOs into the mean, there was an apparent convergence to the heat maps of Fig. 5. In the process, random fluctuating patterns from individual runs disappeared.

For both linear and quadratic moduli we notice that the averaging over NOs preserves the sharp transition between high and low fidelity indices that was characteristic in individual runs. Additionally, we once again observe that the high fidelity index regions in red (with values above 0.99) match well with the expectations from the test dataset for high confidence true predictions based on the mean fidelity index in this range.

For linear (quadratic) moduli, F¯ is plotted on the left (right) heat map of Fig. 5, together with the sinμ=1 and dual bounds. We notice the characteristic concentration of high fidelity values around the sinμ<1 region, which indicates that in the vicinity of this region the NOs correctly recognize predictions with the expected qualitative features of valid solutions.

The heat map of F¯ in the quadratic scan exhibits some additional intriguing features that seem to fit well with features of the heat map derived in Fig. 5 of Ref. [12] using PINNs. The high fidelity red region in our plot stretches above the upper sinμ=1 boundary in a manner that seems to correlate with a region of relatively low-loss solutions [at the orders of O(10-4.5)–O(10-4)] detected by PINNs.⁹

In the heat map of Fig. 5 in [12] the unitarity losses inside the sinμ=1 contour are of the order of 10-8. Here, we are referring to the losses of the PINN solutions right above that region and below the dual bound.

Detecting false predictions. We provided evidence that a high fidelity index (in the interval [0.99, 1.01]) can confidently assess that the prediction is a valid solution. Outside this interval the validity of the prediction is less clear, but it is natural to expect that a low fidelity index will be associated more frequently with a false prediction. We would next like to examine more closely how the fidelity index behaves in situations where the NO fails. For that purpose, we return to the finite partial wave amplitudes with L=4, which proved to be a challenge for the NOs of Sec. IV A 2.

In Fig. 6 we plot the fidelity index and some of the actual predictions from the 56 NOs trained on a combination of true and false samples. The NOs have been evaluated on the modulus of a randomly chosen L=4 amplitude, which is depicted on the top left plot of Fig. 6. On the top right plot, we present the values of the fidelity index for each of the trained NOs. The vast majority of the NOs exhibit a low index with mean value 0.499; this is consistent with the fact that the NOs fail to correctly reproduce the corresponding phase, as one can explicitly check by plotting the predicted output for sinϕ(z) against the exact result. This is useful: the NOs fail to generalize in this case, but they recognize correctly that this is the case and provide a clear indication of that information in the output.

Looking closer at the fidelity indices for each of the 56 NOs in the top right plot of Fig. 6, we also notice that out of the 56 NOs only two have a fidelity index within the interval [0.95, 1]. They are denoted with a red circle in the top right plot of Fig. 6. These two points correspond to fidelity indices 0.957 and 0.962. According to the previous discussion, they lie outside the region that captures a confident true prediction. The predicted sinϕ(z) for these NOs are presented at the bottom plot of Fig. 6 against the exact result, denoted by the thick gray curve, in blue and orange, respectively. The orange prediction, which has the higher fidelity index 0.962, is clearly better and qualitatively closer to an acceptable solution. It is a smoother function within the interval [-1,1] (as one would expect from the sine of a real function), in contrast to the blue prediction that is chaotic and outside the interval [-1,1]. The NO has recognized the importance of these features and has assigned a higher fidelity index to the orange prediction. In principle, it is impossible to exclude the orange prediction as false, but the fact that it stands as a clear outlier in the statistics of Fig. 6, and that the mean fidelity index is very low with small deviation, suggest that the orange prediction is likely false.

In conclusion, the above observations support using NOs within a statistical framework. In general situations, we propose the following approach: When the mean fidelity index suggests that a prediction should be rejected (as in Fig. 6), it should be discarded as potentially false. When the mean fidelity index is high (within the interval [0.99, 1.01]), one should accept the prediction as correct with high probability and extract predictions using the pointwise average of the predicted functions across the collection of NOs. Useful information can also be extracted by the pointwise standard deviation of the predicted functions.¹⁰

A similar statistical approach was also advocated in the optimization schemes of Refs. [38,39]. In that context, the average of independent stochastic optimization runs (especially, those based on reinforcement learning) was always observed to provide better approximations.

Hyperparameters. Following a simple grid search, we observed a significantly larger dependence of the results on the NO hyperparameters for this problem. In what follows, we will report results based on NOs with essentially the same hyperparameters as in Sec. IV A 1. The only hyperparameters that differ are the number of projection channels (we chose 256 instead of 512) and the number of layers (we chose 6 instead of 4). The resulting model has 72,745 parameters.

Training. We attempted training with several types of datasets involving different ratios of unique and ambiguous solutions. Since we were limited to a relatively small range of amplitudes with ambiguous phases at L=2,3, we could not significantly increase the total number of samples, which in turn made the training less efficient. The results presented below are based on a dataset with a total number of 100,000 randomly chosen samples and the following split: 30,000 random L=3 amplitudes assumed to be unique, as well as 10,000 L=2 and 60,000 L=3 amplitudes with ambiguous phases sampled randomly across the different families of solutions summarized in the previous subsection. We trained on 99,000 of these samples and reserved 1000 samples for testing.

We present the results of two, independently trained, NOs with the same hyperparameters, which were trained for 6500 epochs.

Once again, the NOs test well within the training-test dataset. Our purpose here is to explore whether they can achieve any sensible generalization outside their immediate training domain. We will not attempt an exhaustive analysis, opting instead for the study of a few examples for illustration purposes. Specifically, we will focus on the performance of (a) the NOs on the linear and quadratic moduli of Fig. 2 [B(z)=110(z+4) and B(z)=12(z2+1)] that have an infinite partial wave expansion and no phase ambiguities and (b) one of the solutions with phase ambiguities in Ref. [41]—with parameter z1=65+35i—that was also discussed in Ref. [12]; see e.g. Fig. 12 of that paper.

In Fig. 7 we present the predictions of the two NOs for the linear and quadratic moduli. In both cases, the two predicted phases are close to each other and close to the unique exact phase, but the accuracy of the results is obviously lower compared to the results of the previous sections. This is reasonable, since we only trained with 30,000 unique samples (compared to 300,000 unique samples in Sec. IV A 2).

In Fig. 8 we display the corresponding predictions of the two NOs for the Atkinson et al. [41] z1=65+35i modulus. The first NO detects both phases, but the second detects only one of them. More generally, over several runs we observed that properly trained NOs would see either one or both solutions. More frequently, they would detect only one solution (the same one that the second NO detects in Fig. 8).

We also tested the above NOs on the z1=0.31+0.95i ambiguous amplitude of Ref. [12] that has sinμ≃1.67; see Fig. 15 in that paper. The NOs predicted a unique output partially approximating one of the solutions of Ref. [12] with low accuracy. We observed that the prediction was more sensitive (compared to other inputs) to the precise numerics of the input modulus. This is an expected difficulty in general, as it involves generalization to measure-zero configurations and we have no dynamical way to tune the input modulus.

It would be interesting to explore if these problems can be addressed in the following manner: Still within the setting of not invoking the unitarity equation, one could first train a NO (or an ensemble of NOs) to produce a (mean) prediction of two sinϕ and a (mean) fidelity index. Then, in search of ambiguous solutions within the class where the NOs can generalize, one could run a PINN with a NN that models the modulus Bθ(z) (with θ the NN parameters) and a loss function that has two contributions: (i) a repulsive potential for the two sinϕ and (ii) a potential involving the (mean) fidelity index, e.g. of the form (F-1)2. Both contributions are functionals of Bθ(z) and the idea would be to optimize the PINN parameters θ so that it produces moduli with two inequivalent phases and a high fidelity index close to 1.