E. T. Jaynes

Information theory provides a constructive criterion for setting up probability distributions on the basis of partial knowledge, and leads to a type of statistical inference which is called the maximum-entropy estimate. It is the least biased estimate possible on the given information; i.e., it is maximally noncommittal with regard to missing information. If one considers statistical mechanics as a form of statistical inference rather than as a physical theory, it is found that the usual computational rules, starting with the determination of the partition function, are an immediate consequence of the maximum-entropy principle. In the resulting "subjective statistical mechanics," the usual rules are thus justified independently of any physical argument, and in particular independently of experimental verification; whether or not the results agree with experiment, they still represent the best estimates that could have been made on the basis of the information available.It is concluded that statistical mechanics need not be regarded as a physical theory dependent for its validity on the truth of additional assumptions not contained in the laws of mechanics (such as ergodicity, metric transitivity, equal a priori probabilities, etc.). Furthermore, it is possible to maintain a sharp distinction between its physical and statistical aspects. The former consists only of the correct enumeration of the states of a system and their properties; the latter is a straightforward example of statistical inference.

This paper has two purposes: 1) to clarify the relationship between the quantum theory of radiation, where the electromagnetic field-expansion coefficients satisfy commutation relations, and the semiclassical theory, where the electromagnetic field is considered as a definite function of time rather than as an operator; and 2) to apply some of the results in a study of amplitude and frequency stability in a molecular beam maser. In 1), it is shown that the semiclassical theory, when extended te take into account both the effect of the field on the molecules and the effect of the molecules on the field, reproduces almost quantitatively the same laws of energy exchange and coherence properties as the quantized field theory, even in the limit of one or a few quanta in the field mode. In particular, the semiclassical theory is shown to lead to a prediction of spontaneous emission, with the same decay rate as given by quantum electrodynamics, described by the Einstein A coefficients. In 2), the semiclassical theory is applied to the molecular beam maser. Equilibrium amplitude and frequency of oscillation are obtained for an arbitrary velocity distribution of focused molecules, generalizing the results obtained previously by Gordon, Zeiger, and Townes for a singel-velocity beam, and by Lamb and Helmer for a Maxwellian beam. A somewhat surprising result is obtained; which is that the measurable properties of the maser, such as starting current, effective molecular Q, etc., depend mostly on the slowest 5 to 10 per cent of the molecules. Next we calculate the effect of amplitude and frequency of oscillation, of small systematic perturbations. We obtain a prediction that stability can be improved by adjusting the system so that the molecules emit all their energy h Ω to the field, then reabsorb part of it, before leaving the cavity. In general, the most stable operation is obtained when the molecules are in the process of absorbing energy from the radiation as they leave the cavity, most unstable when they are still emitting energy at that time. Finally, we consider the response of an oscillating maser to randomly time-varying perturbations. Graphs are given showing predicted response to a small superimposed signal of a frequency near the oscillation frequency. The existence of "noise enhancing" and "noise quieting" modes of operation found here is a general property of any oscillating system in which amplitude is limited by nonlinearity.

Treatment of the predictive aspect of statistical mechanics as a form of statistical inference is extended to the density-matrix formalism and applied to a discussion of the relation between irreversibility and information loss. A principle of "statistical complementarity" is pointed out, according to which the empirically verifiable probabilities of statistical mechanics necessarily correspond to incomplete predictions. A preliminary discussion is given of the second law of thermodynamics and of a certain class of irreversible processes, in an approximation equivalent to that of the semiclassical theory of radiation.It is shown that a density matrix does not in general contain all the information about a system that is relevant for predicting its behavior. In the case of a system perturbed by random fluctuating fields, the density matrix cannot satisfy any differential equation because $\stackrel{\ifmmode \dot{}\else \.{}\fi{}}{\ensuremath{\rho}}(t)$ does not depend only on $\ensuremath{\rho}(t)$, but also on past conditions The rigorous theory involves stochastic equations in the type $\ensuremath{\rho}(t)=\mathcal{G}(t, 0)\ensuremath{\rho}(0)$, where the operator $\mathcal{G}$ is a functional of conditions during the entire interval ($0\ensuremath{\rightarrow}t$). Therefore a general theory of irreversible processes cannot be based on differential rate equations corresponding to time-proportional transition probabilities. However, such equations often represent useful approximations.

We discuss the relations between maximum-entropy (MAXENT) and other methods of spectral analysis such as the Schuster, Blackman-Tukey, maximum-likelihood, Bayesian, and Autoregressive (AR, ARMA, or ARIMA) models, emphasizing that they are not in conflict, but rather are appropriate in different problems. We conclude that: 1) "Orthodox" sampling theory methods are useful in problems where we have a known model (sampling distribution) for the properties of the noise, but no appreciable prior information about the quantities being estimated. 2) MAXENT is optimal in problems where we have prior information about multiplicities, but no noise. 3) The full Bayesian solution includes both of these as special cases and is needed in problems where we have both prior information and noise. 4) AR models are in one sense a special case of MAXENT, but in another sense they are ubiquitous in all spectral analysis problems with discrete time series. 5) Empirical methods such as Blackman-Tukey, which do not invoke even a likelihood function, are useful in the preliminary, exploratory phase of a problem where our knowledge is sufficient to permit intuitive judgments about how to organize a calculation (smoothing, decimation, windows, prewhitening, padding with zeroes, etc.) but insufficient to set up a quantitative model which would do the proper things for us automatically and optimally.