Construction of constrained experimental designs on finite spaces for a modified Ek-optimality criterion

Dariusz Uciński 1
  • 1 Institute of Control and Computation Engineering, University of Zielona Góra, ul. Szafrana 2, 65-516, Zielona Góra, Poland


A simple computational algorithm is proposed for minimizing sums of largest eigenvalues of the matrix inverse over the set of all convex combinations of a finite number of nonnegative definite matrices subject to additional box constraints on the weights of those combinations. Such problems arise when experimental designs aiming at minimizing sums of largest asymptotic variances of the least-squares estimators are sought and the design region consists of finitely many support points, subject to the additional constraints that the corresponding design weights are to remain within certain limits. The underlying idea is to apply the method of outer approximations for solving the associated convex semi-infinite programming problem, which reduces to solving a sequence of finite min-max problems. A key novelty here is that solutions to the latter are found using generalized simplicial decomposition, which is a recent extension of the classical simplicial decomposition to nondifferentiable optimization. Thereby, the dimensionality of the design problem is drastically reduced. The use of the algorithm is illustrated by an example involving optimal sensor node activation in a large sensor network collecting measurements for parameter estimation of a spatiotemporal process.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Atkinson, A.C., Donev, A.N. and Tobias, R.D. (2007). Optimum Experimental Designs, with SAS, Oxford University Press, Oxford.

  • Beddiaf, S., Autrique, L., Perez, L. and Jolly, J.-C. (2016). Heating source localization in a reduced time, International Journal of Applied Mathematics and Computer Science26(3): 623–640, DOI: 10.1515/amcs-2016-0043.

  • Bernstein, D.S. (2005). Matrix Mathematics. Theory, Facts, and Formulas with Application to Linear Systems Theory, Princeton University Press, Princeton, NJ.

  • Bertsekas, D.P. (1999). Nonlinear Programming, 2nd Edn, Optimization and Computation Series, Athena Scientific, Belmont, MA.

  • Bertsekas, D.P. (2015). Convex Optimization Algorithms, Athena Scientific, Belmont, MA.

  • Bertsekas, D. and Yu, H. (2011). A unifying polyhedral approximation framework for convex optimization, SIAM Journal on Optimization21(1): 333–360.

  • Böhning, D. (1986). A vertex-exchange-method in D-optimal design theory, Metrika33(12): 337–347.

  • Botkin, N.D. and Stoer, J. (2005). Minimization of convex functions on the convex hull of a point set, Mathematical Methods of Operations Research62(2): 167–18.

  • Boyd, S. and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press, Cambridge.

  • Burclová, K. and Pázman, A. (2016). Optimal design of experiments via linear programming, Statistical Papers57(4): 893–910.

  • Chepuri, S.P. and Leus, G. (2015). Sparsity-promoting sensor selection for non-linear measurement models, IEEE Transactions on Signal Processing63(3): 684–698.

  • Coll, C. and Sánchez, E. (2019). Parameter identification and estimation for stage-structured population models, International Journal of Applied Mathematics and Computer Science29(2): 327–336, DOI: 10.2478/amcs-2019-0024.

  • Cook, D. and Fedorov, V. (1995). Constrained optimization of experimental design, Statistics26: 129–178.

  • Djelassi, H., Glass, M. and Mitsos, A. (2019). Discretization-based algorithms for generalized semi-infinite and bilevel programs with coupling equality constraints, Journal of Global Optimization75(2): 341–392.

  • Duarte, B.P.M., Granjo, J.F.O. and Wong, W.K. (2020). Optimal exact designs of experiments via mixed integer nonlinear programming, Statistics and Computing30(1): 93–112.

  • Duarte, B.P.M. and Wong, W.K. (2014). A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models, Statistics and Computing24(6): 1063–1080.

  • Esteban-Bravo, M., Leszkiewicz, A. and Vidal-Sanz, J.M. (2017). Exact optimal experimental designs with constraints, Statistics and Computing27(3): 845–863.

  • Fedorov, V.V. (1989). Optimal design with bounded density: Optimization algorithms of the exchange type, Journal of Statistical Planning and Inference22: 1–13.

  • Fedorov, V.V. and Leonov, S.L. (2014). Optimal Design for Nonlinear Response Models, CRC Press, Boca Raton, FL.

  • Harman, R. (2004). Minimal efficiency of designs under the class of orthogonally invariant information criteria, Metrika60(2): 137–153.

  • Harman, R. and Benková, E. (2017). Barycentric algorithm for computing d-optimal size- and cost-constrained designs of experiments, Metrika80(2): 201–225.

  • Harman, R., Filová, L. and Richtárik, P. (2020). A randomized exchange algorithm for computing optimal approximate designs of experiments, Journal of the American Statistical Association115(529): 348–361.

  • Harman, R. and Pronzato, L. (2007). Improvements on removing nonoptimal support points in d-optimum design algorithms, Statistics & Probability Letters77(1): 90–94.

  • Harville, D.A. (1997). Matrix Algebra From a Statistician’s Perspective, Springer-Verlag, New York, NY.

  • Herzog, R., Riedel, I. and Uciński, D. (2018). Optimal sensor placement for joint parameter and state estimation problems in large-scale dynamical systems with applications to thermo-mechanics, Optimization and Engineering19(3): 591–627.

  • Hettich, R. and Kortanek, K.O. (1993). Semi-infinite programming: Theory, methods and applications, SIAM Review35(3): 380–429.

  • Jacobson, M.Z. (1999). Fundamentals of Atmospheric Modeling, Cambridge University Press, Cambridge.

  • Joshi, S. and Boyd, S. (2009). Sensor selection via convex optimization, IEEE Transactions on Signal Processing57(2): 451–462.

  • Katoh, N. (2001). Combinatorial optimization algorithms in resource allocation problems, in C.A. Floudas and P.M. Pardalos (Eds), Encyclopedia of Optimization, Vol. 1, Kluwer Academic Publishers, Dordrecht, pp. 259–264.

  • Khapalov, A.Y. (2010). Source localization and sensor placement in environmental monitoring, International Journal of Applied Mathematics and Computer Science20(3): 445–458, DOI: 10.2478/v10006-010-0033-3.

  • Langtangen, H.P. and Logg, A. (2016). Solving PDEs in Python. The FEniCS Tutorial I, Springer-Verlag, Cham.

  • Larsson, T., Migdalas, A. and Patriksson, M. (2015). A generic column generation principle: Derivation and convergence analysis, Operational Research15(2): 163–198.

  • Larsson, T., Patriksson, M. and Strömberg, A. (1998). Ergodic convergence in subgradient optimization, Optimization Methods and Software9(1–3): 93–120.

  • Lu, Z. and Pong, T.K. (2013). Computing optimal experimental designs via interior point method, SIAM Journal on Matrix Analysis and Applications34(4): 1556–1580.

  • Maculan, N., Santiago, C.P., Macambira, E.M. and Jardim, M.H.C. (2003). An O(n) algorithm for projecting a vector on the intersection of a hyperplane and a box in ℝn, Journal of Optimization Theory and Applications117(3): 553–574.

  • Marshall, A.W., Olkin, I. and Arnold, B.C. (2011). Inequalities: Theory of Majorization and Its Applications, 2nd Edn, Springer-Verlag, New York, NY.

  • Melas, V. (2006). Functional Approach to Optimal Experimental Design, Springer-Verlag, New York, NY.

  • Patan, M. and Kowalów, D. (2018). Distributed scheduling of measurements in a sensor network for parameter estimation of spatio-temporal systems, International Journal of Applied Mathematics and Computer Science28(1): 39–54, DOI: 10.2478/amcs-2018-0003.

  • Patan, M. and Uciński, D. (2008). Configuring a sensor network for fault detection in distributed parameter systems, International Journal of Applied Mathematics and Computer Science18(4): 513–524, DOI: 10.2478/v10006-008-0045-4.

  • Patan, M. and Uciński, D. (2019). Generalized simplicial decomposition for optimal sensor selection in parameter estimation of spatiotemporal processes, 2019 American Control Conference (ACC), Philadelphia, PA, USA, pp. 2546–2551.

  • Patriksson, M. (2001). Simplicial decomposition algorithms, in C.A. Floudas and P.M. Pardalos (Eds), Encyclopedia of Optimization, Vol. 5, Kluwer Academic Publishers, Dordrecht, pp. 205–212.

  • Pázman, A. (1986). Foundations of Optimum Experimental Design, Mathematics and Its Applications, D. Reidel Publishing Company, Dordrecht.

  • Polak, E. (1987). On the mathematical foundations of nondifferentiable optimization in engineering design, SIAM Review29(1): 21–89.

  • Polak, E. (1997). Optimization. Algorithms and Consistent Approximations, Applied Mathematical Sciences, Springer-Verlag, New York, NY.

  • Pronzato, L. (2003). Removing non-optimal support points in D-optimum design algorithms, Statistics & Probability Letters63: 223–228.

  • Pronzato, L. and Pàzman, A. (2013). Design of Experiments in Nonlinear Models. Asymptotic Normality, Optimality Criteria and Small-Sample Properties, Springer-Verlag, New York, NY.

  • Pronzato, L. and Zhigljavsky, A.A. (2014). Algorithmic construction of optimal designs on compact sets for concave and differentiable criteria, Journal of Statistical Planning and Inference154: 141–155.

  • Pukelsheim, F. (1993). Optimal Design of Experiments, Probability and Mathematical Statistics, John Wiley & Sons, New York, NY.

  • Reemtsen, R. and Görner, S. (1998). Numerical methods for semi-infinite programming: A survey, in R. Reemtsen and J.-J. Rückmann (Eds), Semi-Infinite Programming,Kluwer Academic Publishers, Boston, MA, pp. 195–275.

  • Sagnol, G. (2011). Computing optimal designs of multiresponse experiments reduces to second-order cone programming, Journal of Statistical Planning and Inference141(5): 1684–1708.

  • Sagnol, G. and Harman, R. (2015). Computing exact D-optimal designs by mixed integer second-order cone programming, The Annals of Statistics43(5): 2198–2224.

  • Sahm, M. and Schwabe, R. (2001). A note on optimal bounded designs, in A. Atkinson et al. (Eds), Optimum Design 2000, Kluwer Academic Publishers, Dordrecht, Chapter 13, pp. 131–140.

  • Seber, G.A.F. and Wild, C.J. (1989). Nonlinear Regression, John Wiley & Sons, New York, NY.

  • Shimizu, K. and Aiyoshi, E. (1980). Necessary conditions for min-max problems and algorithms by a relaxation procedure, IEEE Transactions on Automatic ControlAC-25(1): 62–66.

  • Silvey, S.D., Titterington, D.M. and Torsney, B. (1978). An algorithm for optimal designs on a finite design space, Communications in Statistics—Theory and Methods14: 1379–1389.

  • Torsney, B. and Mandal, S. (2001). Construction of constrained optimal designs, in A. Atkinson et al. (Eds), Optimum Design 2000, Kluwer Academic Publishers, Dordrecht, Chapter 14, pp. 141–152.

  • Uciński, D. (2005). Optimal Measurement Methods for Distributed-Parameter System Identification, CRC Press, Boca Raton, FL.

  • Uciński, D. (2012). Sensor network scheduling for identification of spatially distributed processes, International Journal of Applied Mathematics and Computer Science22(1): 25–40, DOI: 10.2478/v10006-012-0002-0.

  • Uciński, D. (2015). An algorithm for construction of constrained D-optimum designs, in A. Steland et al. (Eds), Stochastic Models, Statistics and Their Applications, Springer Proceedings in Mathematics & Statistics, Springer-Verlag, Cham, pp. 461–468.

  • Uciński, D. (2020). D-optimal sensor selection in the presence of correlated measurement noise, Measurement164: 107873.

  • Uciński, D. and Patan, M. (2007). D-optimal design of a monitoring network for parameter estimation of distributed systems, Journal of Global Optimization39(2): 291–322.

  • Wu, C.-F. (1978). Some algorithmic aspects of the theory of optimal designs, The Annals of Statistics6(6): 1286–1301.

  • Yu, Y. (2010). Monotonic convergence of a general algorithm for computing optimal designs, The Annals of Statistics38(3): 1593–1606.

  • Yu, Y. (2011). D-optimal designs via a cocktail algorithm, Statistics and Computing21(3): 475–481.

  • Zarrop, M.B. and Goodwin, G.C. (1975). Comments on “Optimal inputs for system identification”, IEEE Transactions on Automatic ControlAC-20(2): 299–300.

  • Zhang, L., Wu, S.-Y. and López, M.A. (2010). A new exchange method for convex semi-infinite programming, SIAM Journal on Optimization20(6): 2959–2977.


Journal + Issues