Probabilistic & Statistical Aspects of Stochastic Programming

In Norkin (1989, 1992c), Ermoliev and Norkin (1990, 1991) a new concept of normalized convergence of random variables is introduced and investigated. This concept is close to convergence in probability (with some rate) but has some advantages, for instance, it allows to construct confidence intervals for the converging random variables. The following properties of the normalized convergence are established. From a normalized convergence a convergence in probability (with some less rate) follows. Conversely, formally a convergence in probability is a normalized convergence with a degenerated distribution function. From a convergence in mean it follows not only a convergence in probability (that is well known fact) but also a normalized convergence of the corresponding random variables. Normalized convergence is preserved under Hölder transformations of the corresponding random variables. It is also persisted under summation, product, division and direct product of normalized convergent sequences of random variables. Every probabilistic limit theorem implies a normalized convergence of the corresponding random variables. For normalized convergence Cauchy convergence criterion takes place. In Norkin (1989, 1992a), Ermoliev and Norkin (1990, 1991) the concept of normalized convergence is applied to the analysis of convergence of the so-called empirical mean (statistical) method in stochastic optimization.

In Norkin and Roenko (1991), Norkin (1993) certain concavity properties of functions and probability measures are systematically studied, and in Norkin (1993) a stochastic subgradient approach to optimization of probability functions is developed.

In Ermoliev and Norkin (1998) a general stochastic version of the second Lyapunov’s method is developed and applied to establishing some nonstationary laws of large number (with moving expectations) and to proving almost sure convergence of some Monte Carlo optimization and estimation procedures.

In Keyzer, Ermoliev and Norkin (2002) a risk minimization approach to estimate a flexible form that meets a priori restrictions on slope and curvature by means of constraints on both the estimated parameters and the function values is presented. The resulting constrained risk minimization combines parametric and nonparametric estimation and contains integrals and implicit constraints. Within econometrics, simulation has become a common tool to solve problems of this kind. However, it appears that in our case, the simulation approach only applies when the model is linear in parameters, has simple constraints on parameters and a quadratic risk function. To deal with other cases, we use a stochastic optimization technique known as the stochastic quasi-gradient method for stationary and nonstationary problems with Cesàro averaging. This method is also applicable to an expanding series of random observations, and produces asymptotically (weakly) convergent estimates.

Selected references

Norkin V.I. (1992c). On Normalized Convergence of Random Variables, Kibernetika i Sistemnyi Analiz, 1992, N 2, 107-120 (in Russian, English translation in Cybernetics and Systems Analysis, 1992, V. 28, N 2).

Norkin V.I. (1989). Stability of stochastic optimization models and statistical methods of stochastic programming, Preprint 89-53, Glushkov Institute of Cybernetics, Kiev, 1989 (In Russian).

Ermoliev Y.M. and Norkin V.I. (1990). Normalized convergence of random variables and its applications, Kibernetika, 1990, N 6, 85-89. (In Russian, English translation in Cybernetics, 1990, V. 26, N 6).

Ermoliev Y.M. and Norkin V.I. (1991). Normalized convergence in stochastic optimization, Annals of Operations Research, 1991, N 1, 187-198.

Norkin V.I. (1992a). On Convergence and Rate of Convergence of Empirical Mean Value Method in Mathematical Statistics and Stochastic Programming, Kibernetika i Sistemnyi Analiz, 1992, N 3, 84-92 (in Russian, English translation in Cybernetics and Systems Analysis, 1992, V. 28, N 3).

Ermoliev Y.M. and Norkin V.I. (1998). On nonstationary law of large numbers for dependent random variables and its application to stochastic optimization, Kibernetika i sistemny analiz, 1998, N 4, 94-106 (In Russian, English translation in Cybernetics and systems analysis, Vol. 34, N4; see also IIASA Interim Report IR-98-009, Abstract).

Norkin V.I. and Roenko N.V. (1991). a-concave functions and measures and their applications, Kibernatica i Systemnyi Analiz, 1991, N 6, 84-94. (In Russian, English translation in Cybernetics and Systems Analysis, 1991, V. 27, N 6, 860-869).

Keyzer M.A., Ermoliev Y.M., Norkin V.I. (2002). Estimation of econometric models by risk minimization: Stochastic quasi-gradient approach (Abstract), IIASA Interim Report IR-02-021, Int. Inst. for Appl. Syst. Anal., 2002, 32p.