Paper 2022/1699

SoK: Use of Cryptography in Malware Obfuscation

Hassan Asghar, Macquarie University
Benjamin Zi Hao Zhao, Macquarie University
Muhammad Ikram, Macquarie University
Giang Nguyen, Macquarie University
Dali Kaafar, Macquarie University
Sean Lamont, Defence Science and Technology Group
Daniel Coscia, Defence Science and Technology Group
Abstract

We look at the use of cryptography to obfuscate malware. Most surveys on malware obfuscation only discuss simple encryption techniques (e.g., XOR encryption), which are easy to defeat (in principle), since the decryption algorithm and the key is shipped within the program. This SoK proposes a principled definition of malware obfuscation, and categorises instances of malware obfuscation that use cryptographic tools into those which evade detection and those which are detectable. The SoK first examines easily detectable schemes such as string encryption, class encryption and XOR encoding, found in most obfuscated malware. It then details schemes that can be shown to be hard to break, such as the use of environmental keying. We also analyse formal cryptographic obfuscation, i.e., the notions of indistinguishability and virtual black box obfuscation, from the lens of our proposed model on malware obfuscation.

Metadata
Available format(s)
PDF
Category
Foundations
Publication info
Preprint.
Keywords
malware obfuscation environmental keying
Contact author(s)
hassan asghar @ mq edu au
ben_zi zhao @ mq edu au
muhammad ikram @ mq edu au
duclinhgiang nguyen @ hdr mq edu au
dali kaafar @ mq edu au
sean lamont2 @ dst defence gov au
daniel coscia1 @ dst defence gov au
History
2022-12-10: approved
2022-12-07: received
See all versions
Short URL
https://ia.cr/2022/1699
License
Creative Commons Attribution
CC BY

BibTeX

@misc{cryptoeprint:2022/1699,
      author = {Hassan Asghar and Benjamin Zi Hao Zhao and Muhammad Ikram and Giang Nguyen and Dali Kaafar and Sean Lamont and Daniel Coscia},
      title = {SoK: Use of Cryptography in Malware Obfuscation},
      howpublished = {Cryptology ePrint Archive, Paper 2022/1699},
      year = {2022},
      note = {\url{https://eprint.iacr.org/2022/1699}},
      url = {https://eprint.iacr.org/2022/1699}
}
Note: In order to protect the privacy of readers, eprint.iacr.org does not use cookies or embedded third party content.