In this thesis, I propose a method to reduce the cost of computing solutions to optimization problems governed by partial differential equations (PDEs). Standard second order methods such as Newton require the solution of two PDEs per iteration of the Newton system, which can be prohibitively expensive for iterative solvers. In contrast, this work takes advantage a recently developed high order discretization method that comes with an efficient direct solver. The new technique precomputes a solution operator that can be reused for any body load, which is applied whenever a PDE solve is required. Thus the precomputation cost is amortized over many PDE solves. This approach will make second order optimization algorithms computationally affordable for practical applications such as photoacoustic tomography and optimal design problems.