Protections against attacks based on return-oriented programming

• Autor(a) principal: Mateus Felipe Tymburibá Ferreira
• Data de Publicação: 2020
• Tipo de documento: Tese
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/38086
• topic: Return-oriented programming; Protection; Security; Static code analysis; Multilayers; Computação – Teses.; Compiladores (Programas de computador) – Teses.; Programação (Computadores) – Teses.; Algoritmos de computador – Teses.; Programação Orientada a Retornos – Teses..; Computação - Medidas de segurança - Teses.

Return-Oriented Programming (ROP) is the name of a technique used for the development of malicious code that has been widely used to force the execution of arbitrary code on vulnerable applications. Because of its wide use in attacks against modern computing systems, protections against ROP based exploits have been widely studied. Nevertheless, there is still no definitive solution against these attacks. This thesis introduces approaches that hinder the consolidation of such attacks. We created a static source code analysis that estimates the maximum density of indirect branch instructions executable by a program. Its importance stems from the fact that several protections monitor the density of indirect branches to detect ROP attacks. However, these mechanisms use unique thresholds: the frequency of indirect branches that characterizes an attack is the same for any application. In this work, we show that standard thresholds can be easily overcome. As an alternative, we created an algorithm that establishes the best threshold value for each application. Our analysis was implemented in the LLVM compiler and used to find specific thresholds for each benchmark of the SPEC CPU2006 suite. We proved the accuracy of our approach by comparing the estimated thresholds with real values obtained during the execution of these benchmarks in the Pin framework. While statically traversing all possible execution paths of a program is a theoretically undecidable problem in general situations, for the specific context of protection against ROP based attacks we managed to develop an algorithm that finds the solution for programs with up to 700,000 instructions in 25 minutes. In our second approach, we present a multilayer system to guard programs against ROP attacks. Upper layers validate most of a program's control flow at a low computational cost; thus, not compromising runtime. Lower layers provide strong enforcement guarantees to handle more suspicious flows; thus, enhancing security. Our multilayer solution combines techniques already described in the literature with a verification that we introduce: checking if the function call instruction that precedes a return address points to an executable segment of the program. We took care that our project uses micro-architectural units already available in modern versions of x86 processors. Our model imposes a small execution time overhead of 0.57% in a set of benchmarks composed by all 29 programs from SPEC CPU2006 and 209 benchmarks from the LLVM Test Suite. The new check we introduced to validate CALL instructions that precede return addresses is also effective. It reduces by almost 35x the number of gadgets available to build an attack, when compared to a similar technique traditionally employed in the literature. We also analyzed the number of false positives that slip through our layers of protection. In this experiment, we enriched our benchmark suite with the three most popular desktop browsers and four vulnerable applications for which there are known ROP exploits. Only 3.14% of all return addresses are considered invalid in these benchmarks. We demonstrate that the cost to treat these false positives bears no impact in the execution time of programs.