We present a certified reduced basis (RB) framework for the efficient solution of PDE-constrained parametric optimization problems. We consider optimization problems (such as optimal control and optimal design) governed by elliptic PDEs and involving possibly non-convex cost functionals, assuming that the control functions are described in terms of a parameter vector. At each optimization step, the high-fidelity approximation of state and adjoint problems is replaced by a certified RB approximation, thus yielding a very efficient solution through an “optimize-then-reduce” approach. We develop a posteriori error estimates for the solutions of state and adjoint problems, the cost functional, its gradient and the optimal solution. We confirm our theoretical results in the case of optimal control/design problems dealing with potential and thermal flows.