Complex systems appear when interaction among system components creates emergent behavior that is difficult to be predicted from component properties. The growth of Internet of Things (IoT) and embedded technology has increased complexity across several sectors (e.g., automotive, aerospace, agriculture, city infrastructures, home technologies, healthcare) where the paradigm of cyber-physical systems (CPSs) has become a standard. While CPS enables unprecedented capabilities, it raises new challenges in system design, certification, control, and verification. When optimizing system performance computationally expensive simulation tools are often required, and search algorithms that sequentially interrogate a simulator to learn promising solutions are in great demand. This class of algorithms are black-box optimization techniques. However, the generality that makes black-box optimization desirable also causes computational efficiency difficulties when applied real problems. This thesis focuses on Bayesian optimization, a prominent black-box optimization family, and proposes new principles, translated in implementable algorithms, to scale Bayesian optimization to highly expensive, large scale problems. Four problem contexts are studied and approaches are proposed for practically applying Bayesian optimization concepts, namely: (1) increasing sample efficiency of a highly expensive simulator in the presence of other sources of information, where multi-fidelity optimization is used to leverage complementary information sources; (2) accelerating global optimization in the presence of local searches by avoiding over-exploitation with adaptive restart behavior; (3) scaling optimization to high dimensional input spaces by integrating Game theoretic mechanisms with traditional techniques; (4) accelerating optimization by embedding function structure when the reward function is a minimum of several functions. In the first context this thesis produces two multi-fidelity algorithms, a sample driven and model driven approach, and is implemented to optimize a serial production line; in the second context the Stochastic Optimization with Adaptive Restart (SOAR) framework is produced and analyzed with multiple applications to CPS falsification problems; in the third context the Bayesian optimization with sample fictitious play (BOFiP) algorithm is developed with an implementation in high-dimensional neural network training; in the last problem context the minimum surrogate optimization (MSO) framework is produced and combined with both Bayesian optimization and the SOAR framework with applications in simultaneous falsification of multiple CPS requirements.

The energy consumption by public drinking water and wastewater utilities represent up to 30%-40% of a municipality energy bill. The largest energy consumption is used to operate motors for pumping. As a result, the engineering and control community develop the Variable Speed Pumps (VSPs) which allow for regulating valves in the network instead of the traditional binary ON/OFF pumps. Potentially, VSPs save up to 90% of annual energy cost compared to the binary pump. The control problem has been tackled in the literature as “Pump Scheduling Optimization” (PSO) with a main focus on the cost minimization. Nonetheless, engineering literature is mostly concerned with the problem of understanding “healthy working conditions” (e.g., leakages, breakages) for a water infrastructure rather than the costs. This is very critical because if we operate a network under stress, it may satisfy the demand at present but will likely hinder network functionality in the future.

This research addresses the problem of analyzing working conditions of large water systems by means of a detailed hydraulic simulation model (e.g., EPANet) to gain insights into feasibility with respect to pressure, tank level, etc. This work presents a new framework called Feasible Set Approximation – Probabilistic Branch and Bound (FSA-PBnB) for the definition and determination of feasible solutions in terms of pumps regulation. We propose the concept of feasibility distance, which is measured as the distance of the current solution from the feasibility frontier to estimate the distribution of the feasibility values across the solution space. Based on this estimate, pruning the infeasible regions and maintaining the feasible regions are proposed to identify the desired feasible solutions. We test the proposed algorithm with both theoretical and real water networks. The results demonstrate that FSA-PBnB has the capability to identify the feasibility profile in an efficient way. Additionally, with the feasibility distance, we can understand the quality of sub-region in terms of feasibility.

The present work provides a basic feasibility determination framework on the low dimension problems. When FSA-PBnB extends to large scale constraint optimization problems, a more intelligent sampling method may be developed to further reduce the computational effort.