Modeling Non-Stationarity with Deep Gaussian Processes: Applications in Aerospace Engineering

With the rising trend in hierarchical models, Deep Gaussian Processes (DGP) has introduced itself competitive amongst its counterparts by offering a probabilistic framework for deep learning based on the Gaussian Process (GP). An interesting byproduct of DGP, as observed in numerous studies, is its ability to outperform the standard GP with regard to non-stationary functions. Of particular interest to the present study is that of discontinuous-like functions. However, with the current state-of-the-art Doubly Stochastic DGP utilizing a sparse inducing point variational framework, issues regarding its approximate Gaussian posterior and the loss of the interpolation property has propped up in recent works. To that end, a new approach to inference dubbed the DGP stochastic imputation (DGP-SI) has been recently proposed, citing its ability to interpolate. The present study considers the task of surrogate modelling and uncertainty quantification with DGP-SI on three problems with discontinuous-like features compared with a stationary GP. Results indicate that, on average, DGP-SI performs better and shows promise. However, at such an early stage in its development, DGP-SI is sensitive to hyperparameter optimization and thus may result in a model slightly worse than the standard GP.