Architecture for generic detection of machine code injection in computer networks

• Autor(a) principal: Rodrigo Rubira Branco
• Data de Publicação: 2012
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Biblioteca Digital de Teses e Dissertações do ITA
• Texto Completo: http://www.bd.bibl.ita.br/tde_busca/arquivo.php?codArquivo=2116
• topic: Segurança da informação de computadores; Detecção de intrusão (computadores); Arquitetura de software; Controle de acesso; Redes de comunicação; Telecomunicações; Computação

Since the creation of public exploitation frameworks, advanced payloads are used in computer attacks. The attacks generally employ evasion techniques, have anti-forensics capabilities, and use target machines as pivots to reach other machines in the network. The attacks also use polymorphic malicious code that automatically transforms themselves in semantically equivalent variants, which makes them difficult to be detected. The current approaches that try to detect the attacks, fail because they either generate high number of false positives or require high performance capability to work properly. This thesis proposes an architecture for advanced payload detection structured in layers that employ a variant of techniques. The first layers use less computing intensive techniques while the last layers make use of smarter inspection techniques. Fine-grained checks are possible due to the layered approach. For instance, the first layer employs pattern matching while the last layer uses smart network traffic disassembly. In order to improve performance in the detection of forthcoming attacks, the proposed architecture allows the updates of checking rules for more frequently detected attacks. The proposed architecture addresses the high rate false-positives problem using a confidence level updated accordingly to the threat level observed by each layer. We implemented the architecture using real-life workloads and conventional hardware platforms with acceptable throughput. We also contribute with the creation of the well-known Return Address layer optimizing the instruction emulation.