Recommandé
Recommandé
Contenu connexe
Similaire à Learning to Rank Relevant Malware Strings Using Weak Supervision
Similaire à Learning to Rank Relevant Malware Strings Using Weak Supervision (20)
Dernier
Dernier (20)
Learning to Rank Relevant Malware Strings Using Weak Supervision
- 2. ©2019 FireEye©2019 FireEye What is a String? 2 Persists in compilation ASCII/Narrow – N characters + NULL – No file format, context – 0x31 0x33 0x33 0x37 0x00 – ‘1337’, right? – Not necessarily: – Memory addresses – CPU instructions – Data used by the program Unicode/Wide – 2 bytes, double-NULL terminated SourceCode int main() { printf("1337"); return 0; } ObjectFile "1337" .EXEBinary .data 0x56000: "1337"
- 3. ©2019 FireEye©2019 FireEye The Strings Program 3 !This program cannot be run in DOS mode. ??3@YAXPAX@Z ??2@YAPAXI@Z __CxxFrameHandler _except_handler3 WSAStartup() error: %d User-Agent: Mozilla/4.0 (compatible; MSIE 6.00; Windows NT 5.1) GetLastInputInfo SeShutdownPrivilege %sIEXPLORE.EXE SOFTWAREMicrosoftWindowsCurrentVersionApp PathsIEXPLORE.EXE [Machine IdleTime:] %d days + %.2d:%.2d:%.2d [Machine UpTime:] %-.2d Days %-.2d Hours %-.2d Minutes %-.2d Seconds ServiceDll SYSTEMCurrentControlSetServices%sParameters if exist "%s" goto selfkill del "%s" attrib -a -r -s -h "%s" Inject '%s' to PID '%d' Successfully! cmd.exe /c Hi,Master [%d/%d/%d %d:%d:%d]
- 4. ©2019 FireEye©2019 FireEye Malware Triage 4 Customer Suspected compromise Incident Response Forensic analysis Identify malware sample Reverse Engineer Binary triage Malware analysis +SOC analysts, red teamers, malware researchers, n00bs, experts
- 5. ©2019 FireEye©2019 FireEye Strings in Practice – Static Analysis 5 Running Strings on larger binaries produces tens of thousands of strings. Strings produces a ton of noise mixed in with important information Knowing which strings are relevant often requires highly experienced analysts. Relevance is subjective and its definition can vary significantly across analysts.
- 6. ©2019 FireEye©2019 FireEye Hypothesis and Goals 6 Develop a tool that can: – efficiently identify and prioritize strings – based on relevance for malware analysis StringSifter should: – be easy to use – generalize across: – roles, use cases, downstream apps – save time and money How does it work?
- 7. ©2019 FireEye©2019 FireEye ( ) Create optimal ordering of a list of items Precise individual item scores less important than their relative ordering In classification, regression, clustering we predict a class or single score LTR rarely applied in security applications Learning to Rank 7 f
- 8. ©2019 FireEye©2019 FireEye Rank items within unseen lists in a similar way to rankings within training lists Each item associated with a set of features and an ordinal integer label Ordinal label is the teaching signal that encodes relevance level LTR as Supervised Learning 8
- 9. ©2019 FireEye©2019 FireEye EMBER Dataset 9 Endgame Malware BEnchmark for Research – v1 (1.1 million PE files scanned on or before 2017) https://arxiv.org/abs/1804.04637 https://github.com/endgameinc/ember – 400k train + test malware binaries from v1 malware defined as > 40 VT vendors say malicious Ran Strings on 400k malware binaries – produced 3+ billion ASCII + Unicode strings (24+ GB) – performed sampling, stratified by malware family – labeled according to weak supervision
- 10. ©2019 FireEye©2019 FireEye Weak Supervision 10 https://github.com/snorkel-team/snorkel Data Labeling Bottleneck Ordinal Labeling Functions – ABSTAIN = -1 – VERY_IRRELEVANT = 0 – IRRELEVANT = 1 – SEMI_IRRELEVANT = 2 – NEUTRAL = 3 – SEMI_RELEVANT = 4 – RELEVANT = 5 – VERY_RELEVANT = 6 cardinality = 7, tie goes NEUTRAL Apply 70+ LFs over input strings, generate probabilistic labels
- 11. ©2019 FireEye©2019 FireEye Natural Language Processing – Entropy rate – KL Divergence – Markov model Host, Network IoCs Malware Regexes – encodings (base64) – format specifiers – user agents LF Examples 11 t % F 0.02 0.07 0.01 0.2 0.2 0.01 0.03 0.14 0.05 0.01 http://evil.com SOFTWAREincludeevil.pdb t%Ft Vr}Y 0.018 0.014 0.007 0.001 [(‘QQQQ’, 0), (’QQQQ’, 0), (’PPPPP’, 0), (’PPPPP’, 0), (’WWWWW’, 0), (‘SSSSS’, 0), (’SSSSS’, 0), … (’VVVVV’, 0)] [(‘~!@#$%^&*()_+`1234567890-=qwertyuiop[]QWERTYUIOP{}|asdfghjkl;`ASDFGHJKL:”zxcvbnm,./ZXCVBNM<> ?’,4.55), (‘!”#$%&’()*+,-./0123456789:;?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~’, 4.54), (‘!”#$%&’()*+,-./0123456789:;?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~’, 4.54), … (‘!”#$%&’()*+,-./0123456789:;?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~’, 4.54)] [(‘ttt’, 0), (’_&t`’, 0), (’toti’, 0), (’at=Q’, 0), … (’oti0{’, 0)] [(‘winter’, 0.15), (‘Finger’, 0.16), (‘Inter’, 0.17), … (’Conte’, 0.18)]
- 13. ©2019 FireEye©2019 FireEye Learning a LabelModel 13 https://github.com/snorkel-team/snorkel LFS have different: – Accuracies – Correlations – Certainties Learning a LabelModel – Inverse generalized covariance matrix of LFs – Matrix completion (Robust PCA) Snorkel @ ICML ’19 – https://arxiv.org/abs/1903.05844
- 14. ©2019 FireEye©2019 FireEye Gradient Boosted Decision Trees (GBDTs) – combine outputs from multiple Decision Trees – reduce loss using gradient descent – weighted sum of trees’ predictions as ensemble – LightGBM (https://github.com/microsoft/LightGBM) Histogram-binned GBDTs with LTR obj. function Neural networks – tf-ranking (https://github.com/tensorflow/ranking) – Scoring Function: defines the network – Loss (e.g. pairwise logistic), Metrics (e.g. precision) LTR Models 14
- 15. ©2019 FireEye©2019 FireEye Normalized Discounted Cumulative Gain – Normalized: divide DCG by ideal DCG on a ground truth holdout dataset – Discounted: divides each string’s predicted relevance by a monotonically increasing function (log of its ranked position) – Cumulative: the cumulative gain or summed total of every string’s relevance – Gain: the magnitude of each string’s relevance Evaluation 15
- 16. ©2019 FireEye©2019 FireEye Kernel Density of Test NDCG Scores 16 StringSifter performs well on a holdout set of 7+ years of FLARE malware reports.
- 17. ©2019 FireEye©2019 FireEye Putting it All Together 17
- 19. ©2019 FireEye©2019 FireEye Open Sourcing StringSifter 19 The tool is now live – Command line and Docker tools – flarestrings <my_sample> | rank_strings FLOSS outputs, live memory dumps Weak Supervision for Cybersecurity – Other label-starved problems? In the works – more labeling functions, mach-o + ELF files https://github.com/fireeye/stringsifter pip install stringsifter @phtully
Notes de l'éditeur
- Starts at hex 21 / 94 printable characters
- Traditional ML solves a prediction problem (classification or regression) on a single instance at a time. E.g. if you are doing spam detection on email, you will look at all the features associated with that email and classify it as spam or not. The aim of traditional ML is to come up with a class (spam or no-spam) or a single numerical score for that instance. LTR solves a ranking problem on a list of items. The aim of LTR is to come up with optimal ordering of those items. As such, LTR doesn’t care much about the exact score that each item gets, but cares more about the relative ordering among all the items.
- Traditional ML solves a prediction problem (classification or regression) on a single instance at a time. E.g. if you are doing spam detection on email, you will look at all the features associated with that email and classify it as spam or not. The aim of traditional ML is to come up with a class (spam or no-spam) or a single numerical score for that instance. LTR solves a ranking problem on a list of items. The aim of LTR is to come up with optimal ordering of those items. As such, LTR doesn’t care much about the exact score that each item gets, but cares more about the relative ordering among all the items.
- Traditional ML solves a prediction problem (classification or regression) on a single instance at a time. E.g. if you are doing spam detection on email, you will look at all the features associated with that email and classify it as spam or not. The aim of traditional ML is to come up with a class (spam or no-spam) or a single numerical score for that instance. LTR solves a ranking problem on a list of items. The aim of LTR is to come up with optimal ordering of those items. As such, LTR doesn’t care much about the exact score that each item gets, but cares more about the relative ordering among all the items.
- Traditional ML solves a prediction problem (classification or regression) on a single instance at a time. E.g. if you are doing spam detection on email, you will look at all the features associated with that email and classify it as spam or not. The aim of traditional ML is to come up with a class (spam or no-spam) or a single numerical score for that instance. LTR solves a ranking problem on a list of items. The aim of LTR is to come up with optimal ordering of those items. As such, LTR doesn’t care much about the exact score that each item gets, but cares more about the relative ordering among all the items.
- Traditional ML solves a prediction problem (classification or regression) on a single instance at a time. E.g. if you are doing spam detection on email, you will look at all the features associated with that email and classify it as spam or not. The aim of traditional ML is to come up with a class (spam or no-spam) or a single numerical score for that instance. LTR solves a ranking problem on a list of items. The aim of LTR is to come up with optimal ordering of those items. As such, LTR doesn’t care much about the exact score that each item gets, but cares more about the relative ordering among all the items.
- - Learning to rank learns to directly rank items by training a model to predict the probability of a certain item ranking over another item. - This is done by learning a scoring function where items ranked higher should have higher scores. The model can be trained via gradient descent on a loss function defined over these scores. - For each item, gradient descent pushes the score up for every item that ranks below it and pushes the score down for every item that ranks above it. The “strength” of the push is determined by the difference in scores. - To ensure that the model focuses on getting the higher ranks (which are generally more important) correct, we can weight the “strength” of the push by a factor that accounts for how important the ranking is.
- Discounted reflects the goal of having the most relevant strings ranked towards the top of our predictions Normalization makes it possible to compare scores across samples since the number of strings within different Strings outputs can vary widely. which we obtain from FLARE-identified relevant strings contained within historical malware reports.
- Discounted reflects the goal of having the most relevant strings ranked towards the top of our predictions Normalization makes it possible to compare scores across samples since the number of strings within different Strings outputs can vary widely. which we obtain from FLARE-identified relevant strings contained within historical malware reports.