The use of reactive security mechanisms in enterprise networks can, at times, provide an asymmetric advantage to the attacker. Similarly, the use of a proactive security mechanism like Moving Target Defense (MTD), if performed without analyzing the effects of security countermeasures, can lead to security policy and service level agreement violations. In this thesis, I explore the research questions 1) how to model attacker-defender interactions for multi-stage attacks? 2) how to efficiently deploy proactive (MTD) security countermeasures in a software-defined environment for single and multi-stage attacks? 3) how to verify the effects of security and management policies on the network and take corrective actions?

I propose a Software-defined Situation-aware Cloud Security framework, that, 1) analyzes the attacker-defender interactions using an Software-defined Networking (SDN) based scalable attack graph. This research investigates Advanced Persistent Threat (APT) attacks using a scalable attack graph. The framework utilizes a parallel graph partitioning algorithm to generate an attack graph quickly and efficiently. 2) models single-stage and multi-stage attacks (APTs) using the game-theoretic model and provides SDN-based MTD countermeasures. I propose a Markov Game for modeling multi-stage attacks. 3) introduces a multi-stage policy conflict checking framework at the SDN network's application plane. I present INTPOL, a new intent-driven security policy enforcement solution. INTPOL provides a unified language and INTPOL grammar that abstracts the network administrator from the underlying network controller's lexical rules. INTPOL develops a bounded formal model for network service compliance checking, which significantly reduces the number of countermeasures that needs to be deployed. Once the application-layer policy conflicts are resolved, I utilize an Object-Oriented Policy Conflict checking (OOPC) framework that identifies and resolves rule-order dependencies and conflicts between security policies.

As automation research into penetration testing has developed, several methods have been proposed as suitable control mechanisms for use in pentesting frameworks. These include Markov Decision Processes (MDPs), partially observable Markov Decision Processes (POMDPs), and POMDPs utilizing reinforcement learning. Since much work has been done automating other aspects of the pentesting process using exploit frameworks and scanning tools, this is the next focal point in this field. This paper shows a fully-integrated solution comprised of a POMDP-based planning algorithm, the Nessus scanning utility, and MITRE's CALDERA pentesting platform. These are linked in order to create an autonomous AI attack platform with scanning, planning, and attack capabilities.

Vulnerability testing/evaluation is a regular task for cyber-security groups. Conducting tasks like this can take up a great amount of time and may not be perfect. Automating these tasks helps speed up the rate at which experts can test systems. However, script based or static programs that run automatically often do not have the versatility required to properly replace human analysis. With the advances in Artificial Intelligence and Machine Learning, a utility can be developed that would allow for the creation of penetration testing plans rather than manually testing vulnerabilities. A variety of existing cyber-security programs and utilities provide an API layer that commonly interacts with the Python environment. With the commonality of AI/ML tools within the Python ecosystem, a plugin like interface can be developed to feed any AI/ML program real world data and receive a response/report in return. Using Python 2.7+, Python 3.6+, pymdptoolbox, and POMDPy, a program was developed that ingests real-world data from scanning tools and returned a suggested course of action to be used by analysts in order to perform a practical validation of the algorithms in a real world setting. This program was able to successfully navigate a test network and produce results that were expected to be found on the target machines without needing human analysis of the network. Using POMDP based systems for more cyber-security type tasks may be a valuable use case for future developments and help ease the burden faced in a rapid paced world.

The increasing usage of smart-phones and mobile devices in work environment and IT

industry has brought about unique set of challenges and opportunities. ARM architecture

in particular has evolved to a point where it supports implementations across wide spectrum

of performance points and ARM based tablets and smart-phones are in demand. The

enhancements to basic ARM RISC architecture allow ARM to have high performance,

small code size, low power consumption and small silicon area. Users want their devices to

perform many tasks such as read email, play games, and run other online applications and

organizations no longer desire to provision and maintain individual’s IT equipment. The

term BYOD (Bring Your Own Device) has come into being from demand of such a work

setup and is one of the motivation of this research work. It brings many opportunities such

as increased productivity and reduced costs and challenges such as secured data access,

data leakage and amount of control by the organization.

To provision such a framework we need to bridge the gap from both organizations side

and individuals point of view. Mobile device users face issue of application delivery on

multiple platforms. For instance having purchased many applications from one proprietary

application store, individuals may want to move them to a different platform/device but

currently this is not possible. Organizations face security issues in providing such a solution

as there are many potential threats from allowing BYOD work-style such as unauthorized

access to data, attacks from the devices within and outside the network.

ARM based Secure Mobile SDN framework will resolve these issues and enable employees

to consolidate both personal and business calls and mobile data access on a single device.

To address application delivery issue we are introducing KVM based virtualization that

will allow host OS to run multiple guest OS. To address the security problem we introduce

SDN environment where host would be running bridged network of guest OS using Open

vSwitch . This would allow a remote controller to monitor the state of guest OS for making

important control and traffic flow decisions based on the situation.