Recommended
More Related Content
What's hot
What's hot (20)
Similar to Evaluation of Web Application Vulnerability Scanners' Strengths and Weaknesses Using a Custom Test Bed
Similar to Evaluation of Web Application Vulnerability Scanners' Strengths and Weaknesses Using a Custom Test Bed (20)
Recently uploaded
Recently uploaded (20)
Evaluation of Web Application Vulnerability Scanners' Strengths and Weaknesses Using a Custom Test Bed
- 1. EVALUATION OF WEB APPLICATION VULNERABILITY SCANNERS’ STRENGTHS AND LIMITATIONS USING CUSTOM WEB APPLICATION By: Yuliana Martrosyan Advisor: Dr. Levent Ertaul
- 2. GOAL OF THE THESIS How efficient WAVS are to address security concerns in the web applications? Develop custom test bed that implements vulnerabilities presented in the web Assess results to suggest areas that require research to improve WAVS detection rate
- 3. OWASP TOP 10 PROJECT
- 4. ACUNETIX WVS
- 5. QUALYS WAS
- 6. Modeling User Behavior Create Account Update Account Create Shopping Cart Check Product Review Add Product Review Recover Password Partners’ Newsletters Subscribe to Mailing List MusicStore implements 55 variations of OWASP Top 10 Vulnerabilities
- 7. First Order SQLI String query = ” SELECT Password FROM v_UserPass WHERE v_UserPass.EmailAddress = ’” + emailAddress + "’ AND v_UserPass.Answer = ’”+ answer +"') ";
- 8. First Order SQLI Acunetix WVS Missed all SQLI. Not all of the required fields were filled. QualysGulard WAS First-Order SQLI was detected
- 9. Second Order SQLI String query = "UPDATE v_UserPass SET ” + "Password = ?” + ", Answer = ” + "’”+ answer + "' ”+ ” WHERE EmailAddress = '” + emailAddress + "'";
- 10. Second Order SQLI Both Acunetix WVS and QualysGulard WAS Missed all Second Order SQLI
- 11. Cross-Site Scripting XSS Acunetix WVS Missed Persistent XSS. Unable to find the payload QualysGuard WAS Detected most Non- Persistent, Persistent.
- 12. DOM XSS Acunetix WVS Detected all AJAX XSS and most Non- Persistent XSS QualysGuard WAS Missed all AJAX vulnerabilities. Detected some other DOM vulnerabilities. GET Request: http://vulnerablewebapp.com/email/addToE mailList? firstName=%3CIFRAME%20src=javascri pt:alert(%27firstName%20XSS%27)%20/ %3E&lastName=Simpson&emailAddress= hs@hs .com
- 13. Broken Authentication and Session Management Both scanners missed Weak Password Recovery Model vulnerability. Both scanners found Brute Force attack vulnerability.
- 14. Both scanners detected INSECURE DIRECT OBJECT REFERENCE
- 15. Cross Site Request Forgery (CSRF) Most vulnerabilities were missed by both scanners due to incomplete crawling phase. QualysGuard WAS reported numerous CSRF duplicated marked as ‘clickjacking’ vulnerabilities
- 16. SECURITY MISCONFIGURATION QualysGuard detected DoS attack possibility. Data submission HTTP methods should be tested.
- 17. INSECURE CRYPTOGRAPHIC STORAGE AND INSUFFICIENT TRANSPORT LAYER PROTECTION Non-Encrypted Storage Displaying passwords while typing No ‘secure’ and ‘HTTPOnly’ cookies No SSL with Log In No SSL with Confidential Info Both scanners recommend setting ‘secure’ flag to the application cookies. Both scanners should test for insecure handling of confidential data
- 18. FAILURE TO RESTRICT URL ACCESS Both scanners did not detect the hidden link. Advanced force browsing should be performed
- 19. UN-VALIDATED REDIRECT AND FORWARD QualysGuard WAS found the flaw. Acunetix WVS should spider the site to see if it generates any redirects.
- 20. DETECTED AND FALSE POSITIVES QualysGuard WAS Acunetix WVS 0 20 40 60 80 100 Detected FP 0 50 100 150 200 250 Detected FP
- 21. GROUP RESULTS 0 20 40 60 80 100 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 DetectionRate OWASP Vulnerabilities
- 22. PUBLICATIONS L. Ertaul Y. Martirosyan Implementation of a WEB Application for Evaluation of WEB Application Security Scanners Proceedings of the 2012 International Conference on Security & Management SAM’12, July, Las Vegas.
- 23. CONCLUSION Improve Crawling Functionality Re-indexing the pages of target application after the attack to detect the payload Check all possible attack vectors and then report the vulnerability and reproduction steps Use WASSs as a group
Editor's Notes
- The Open Web Application Security Project (OWASP) security community has released its annual report capturing the top vulnerabilities and risks in web application development as a combination of the probability of an event and its consequence. A1 Injection. Introducing a malicious data into a computer program causes an injection attack. Malicious data can enter the program at specific places and later is exploited by an attacker. Many types of vulnerabilities, including SQL Injection (SQLI), belong to the general class of injection flaws. SQLI vulnerability occurs when there is a possibility to trick the SQL engine into executing unintended commands. SQLI vulnerabilities are exploited using SQLI attacks. SQLI attacks are usually divided into three categories: First Order SQLI Attack, Second Order or Blind SQLI Attack, and Database Constants SQLI Attack. Apart from SQLI, there are other prominent examples for injection vulnerabilities: XML injection, OS commands injection, SSI injection. In this thesis, SQLI vulnerability type is focused because it occurs more frequently in real-world applications than the other types of Injection vulnerability. A2 XSS. Cross Site Scripting (XSS) vulnerability occurs when there is a possibility of injection of malicious code in web application. Thus, the XSS flaw is as a result of not validated or sanitized input parameters. There are three types of XSS: Non-Persistent, sometimes also called Reflected XSS, Persistent or Stored XSS and Document Object Model (DOM) –based. A3 Broken Authentication. The user authentication on the web typically involves the use of a user’s ID and password. Stronger methods of authentication are commercially available, such as software and hardware based cryptographic tokens or biometrics. But these mechanisms are cost prohibitive for most web applications. When authentication mechanism does not provide enough protection, an attacker can try to obtain credentials by using different techniques or some combination. A4 Insecure Direct Object Reference. A situation when files, directories, and database records are exposed to user. A5 CSRF. CSRF attacks have been called the ‘sleeping giant’ of web-based vulnerabilities. CSRF vulnerability occurs when an attacker can force a victim’s web browser to make a request to a website of the attacker’s choosing. A6 Security Misconfiguration. This type of vulnerability occurs when application, frameworks, application server, web server, database server, and platform configurations are not securely defined to prevent unintentional leakage of information. A7 Insecure Cryptographic Storage. This vulnerability occurs when web application is failing to encrypt sensitive data. It can be broken down to two main areas: Encryption and Hashing. A8 Failure to Restrict URL Access. This vulnerability usually occurs when unauthorized users are able to access content of web pages that are intended to be used by users with special privileges, for example administrators. In 2007, the Macworld Conference & Expo web site failed to restrict special URL access to a Steve Jobs keynote speech and let users get “Platinum” passes worth nearly $1,700, all for free. A9 Insufficient transport layer protection. This vulnerability occurs as a result of lack of transport layer encryption, weak cipher support, or not having efficient protection of sensitive network traffic. A10 Un-validated Redirect and Forward. Insecure implementation of Redirect and Forward functionality can result in tricking the user into clicking the link that will navigate to an unsafe destination.
- In this thesis an experiment was conducted by running QualysGuard WAS and Acunetix WVS against our test suit. Both Scanners Support: Detection of vulnerabilities listed in OWASP Top Ten report Authentication scheme JavaScript AJAX DOM Acunetix Web Vulnerability Scanner (WVS) is an automated web application security-testing tool that audits web applications by checking for vulnerabilities like SQL Injections, Cross-Site Scripting and other exploitable hacking vulnerabilities. In general, Acunetix WVS scans any website or web application that is accessible via a web browser and uses the HTTP/HTTPS protocol. Acunetix WVS works in the following manner: 1. Crawl 2. Attack 3. Port Scan 4. Report vulnerability in ‘Alert’ node 5. Report open ports to ‘Knowledge Base’ node 6. Create Scan Report a shown in the slide
- QualysGuard Web Application Scanning (WAS) is a web application vulnerability scanner that identifies web application vulnerabilities in the OWASP Top Ten report, like SQL injection, cross-site scripting (XSS) URL redirection, and others. The tool allows users to: Crawl web applications and scan them for vulnerabilities. Identify web applications’ handling of sensitive or secret data. Customize: authentication, black/white lists, robots.txt, sitemap.xml and more. View reports with recommended security coding practice and configuration.
- MusicStore is web-based online store application fully simulates publicly available online stores’ functionality. Each action on the web site can be seen as real-life user behavior on a typical web commerce application. First a user creates an account, providing his/her personal data, including credit card number and shipping address. Second he/she selects the product and stores his selection in personal shopping cart. Later when the user decides to make the purchase an invoice is placed in queue for further processing. User has total control over his account and can make any changes in his personal settings, including updating personal data and credentials, and even recover forgotten password. In addition to that the user can add reviews to products and read other customers’ opinions, check partners’ newsletters and subscribe to mailing list.
- MusicStore web application contains two First Order SQLI examples. For example, the password recovery functionality can be exploited by modifying the SQL query. The recoverPassword(String emailAddress, String answer) function is intended to recover user’s password based on the answer to a security question. In recoverPassword function, concatenation is used to create a dynamic SQL query. An attacker can easily impersonate a site user, e.g. ‘test@test.com’, and recover a victim’s password by commenting out the part of the query using ‘--’ single-line comment indicator.
- An attacker can try to find out, whether a user with ‘test1@test.com’ email address exists in the database. To do that, the attacker uses blind SQLI on ‘answer’ parameter, trying true and false payloads. Injecting the true payload, the account password will be updated; True Payload: password=test11&answer=red%27+WHERE+EmailAddress%3D%28%27test1%40test.com%27%29 --
- Injecting the false payload, attacker will see an error message. False Payload: password=test11&answer=red%27+WHERE+EmailAddress%3D%28%27emailnotexist%27%29 -- both WAVSs failed to find Second Order SQL Injection vulnerabilities. This may be because of the essence of Second Order SQLI: the payload is not executed immediately. The result of the SQLI is displayed on a page that should be navigated by user after the payload was submitted. WAVSs fail to follow this logic, thus interpreting it as a negative response.
- A customer can add his/her review to the product on special page, where only registered users have access. insert(Review review) function handles this functionality. String query = "INSERT INTO v_Reviews (ProductID, UserID, ReviewDate, Title, Message) VALUES ("+ productID + ", "+ userID + ", SYSDATE, + title + , + message+ )"; The attacker tampers the HTTP request to have XSS payload in it. Vulnerability: /user/review/displayReview Payload: title=Title+%3Cscript%3Ealert%280%29%3C%2Fscript%3E&message=Message&SUBMIT=Submit The result of the payload execution is not displayed at the same page, where the malicious code was injected. The payload is stored in the database, and later is executed on the page where all customers can view the reviews Non-Persistent and Persistent Acunetix WVS Missed Persistent XSS. Unable to find the payload QualysGuard WAS Detected most Non-Persistent, Persistent. The experiment suggests, that the injected patterns can overwrite formerly injected patterns before they are detected by the analyzer component. In order to increase the detection rate of XSS vulnerabilities, particularly Persistent XSS flaws, the pages of an application should be re-indexed after the attack.
- In this DOM XSS example, XmlHTTPRequest (AJAX technology) is used. ‘First name’, ‘Last name’, ‘Email address’ fields’ values are used to add a user to a mailing list. When a user enters these values, the result is displayed on the same web page without refreshing the entire page. Payload: http://134.154.14.153:8080/yuliana/email/addToEmailList?firstName=%3CIFRAME%20src=javascript:alert(%27firstName%20XSS%27)%20/%3E&lastName=Simpson&emailAddress=hs@hs .com The XSS payload in ‘firstName’ parameter can use AJAX requests to autonomously inject itself into pages and easily re-inject the same host with more XSS, all of which can be done with no hard refresh as shown in Figure 3.13. The same XHR XSS vulnerability is present for ‘lastName’ and ‘emailAddress’ parameters. WAVSs should implement more modern techniques for crawling to avoid missing pages that are using AJAX.
- We present two types of Broken Authentication vulnerabilities. The first one can be exploited using social engineering, which allows the attacker to guess the possible user secret by tricking the user into revealing his/her personal information. The recovery function is based on the security question. Vulnerability: Question. Where were you born? Attacker can trick user into revealing his/her place of birth, simply asking: ”Where are you from” The second type is Brute Force attack. Broken Authentication and Session Management Guessing (Weak Password Recovery Model vulnerability), as long as other social engineering techniques are straightforward for a human user, but they represent a challenge for automated tools. As a result, the scanners were not able to find the flow, which is not surprising. Both WAVSs easily discovered the second vulnerability because it had plain brute force attack possibility. This is because the login brute force option is included in default settings of tested WAVSs.
- the web application receives reference to a file as a form parameter ‘letter’, and then reads and displays the text. The web application has a number of partners that have their own web pages. Payload: ../../../../../../../apps/java/apache-tomcat-6.0.16/conf/server.xml Payload Reflects: in ‘partnerText’ div an attacker could tamper with ‘letter’ parameter value in HTTP Request to access server.xml file of the Tomcat Apache server. Insecure Direct Object Reference Both scanners detected. For Insecure Direct Object Reference vulnerability type, it is crucial to discover the vulnerable parameter, because by manipulating its value, an attacker can access the web pages outside the allowed directory.
- The attacker can change a victim’s personal information, including shopping address, credit card number, and password. For example, only a logged in user can modify his/her invoice shipping address. An attacker ‘test1@test.com’ can obtain the JSESSIONID cookie of a legitimate user ‘test@test.com’ and impersonate the customer CSRF The experiment suggested that the main reason CSRF vulnerability type has so many undiscovered vulnerabilities is that the tools didn’t have good in-depth coverage of the MusicStore. Thus the crawling functionality should be enhanced. The general recommendation to prevent CSRF duplicates is to avoid separation of clickjacing and CSRF attacks. But the decision, whether the clickjacing should be considered as separate threat is individual for each scanner.
- All requests that contain confidential information, like credit card number or password, should be handled using POST method. If the form that contains confidential information can be submitted by GET method, then an attacker can trick a victim to change his/her confidential information without being aware of that fact. An attacker can place a hidden link on an email address, asking to visit a new online shop. User will click ‘Visit us’ web page, but instead of seeing a new ecommerce web site, his/her password will automatically be changed. Security Misconfiguration The 2 vulnerabilities missed by the QualysGuard WAS in this type are based on insecure data handling by web server, which is able to process requests sent by GET method. Scanners missed this vulnerability because the form with sensitive data was submitted by POST method although it was possible to send the request by adding the parameters in URL and process it as GET method. Apparently, the testing of the request transfer method is not even included in the tools functionality.
- Insecure Cryptographic Storage The scanners recommend setting ‘secure’ flag to the application cookies. Although in general this recommendation is useful, it doesn’t make sense if an application doesn’t use HTTPS. The scanners should search for keywords, indicating a confidential data, for example, ‘password’, ‘credit card’ and ‘secret’. Insufficient Transport Layer Protection The scanners were able to detect all insecure cookie and session processing vulnerabilities. To improve the results, WAVSs should pay more attention to non-encrypted connections while handling confidential data.
- MusicStore protects all data under ‘/user’ directory. After a user is authenticated, web application grants him/her an access to a hidden ‘userAccess.jsp’ web page. But, ‘userAccess.jsp’ is not under ‘/user’ directory. Thus, an attacker can guess this hidden link by using crawling tools and take advantage. JSP expression language code, that checks if a customer is logged in, is vulnerable and doesn’t provide required restriction to URL access. Failure to Restrict URL Access Both scanners did not detect the hidden link. The link is accessible by registered user only. Another way to reach the hidden link is force browsing which has failed for scanner
- The web application has a number of partners that have their own web pages. Each of the partners has their link on MusicStore. The link “Visit us” takes the customer to ‘8AM’ partner’s web site, ‘www.example.com’. This is called un-validated redirection. Un-validated Redirect and Forward Acunetix WVS didn’t report any findings. In order to avoid these shortcomings, Acunetix WVS should spider the site to see if it generates any redirects. Next it should check the parameters supplied prior to the redirect to see if they appear to be a target URL or a piece of such a URL. If so, change the URL target and observe whether the site redirects to the new target. Or check all parameters to see if they look like part of a redirect or forward URL destination. The link is accessible by registered user only. Another way to reach the hidden link is force browsing which has failed for WAVS
- MusicStore containes the secure part, where the defense mechanisms against OWASP vulnerability types are implemented. Its significance is observed while analyzing the False Positive (FP) results obtained by running WAVS. FP SQLI is the result of improper Blind SQLI technique, implemented by the Acunetix WVS. The was inserted in the database using java Prepared Statement thus there is no need in escaping it. After the attack, the application returned different HTML page, so the scanner decided, that the attack was successful. In reality, the payload was never executed and was displayed later, just like an ordinary text using. Acunetix WVS should test for false payload. The result of injecting this payload is the same, as which injecting the true payload, described earlier, thus this is not a real vulnerability. XSS FP Acunetix WVS FP rate for XSS vulnerabilities is 205.5. This is because the scanner reported the same field value being vulnerable multiple times. Acunetix WVS recognized that a filed value was vulnerable with one set of parameters, but then tested it again by changing some of the other parameters on the page. The same with QualysGuard WAS. CSRF FP The interesting result for Scanner Q was found for CSRF vulnerability type as shown in Fig. 4. False Positive rate is higher than Detected. This means that despite the fact that scanner is very attentive to this type of weaknesses and suspected many web pages to be vulnerable it wasn’t able to reach all possible web pages to try there the attacks as a result of complex multi-step application design.
- As a group two scanner showed significantly better results, in particular almost All XSS vulnerabilities were detected (94.5%)
- Our finding were accepted SAM’12 Worldcomp conference. And were published in “Implementation of a web application for evaluation of web application security scanners” paper in Las Vegas on July 2012. Currently the paper has three referees: the scanner’s research labs and independent security analytic.
- Improve Crawling Functionality to increase Detection Rate of vulnerabilities, such as AJAX XSS, CSRF, etc. The pages of a target application should be re-indexed after the attack to increase Detection Rate of Stored vulnerability types, such as Stored SQL, Persistent XSS Check all possible attack vectors and then report the vulnerability once to avoid ‘Duplicate’ results and decrease False Positive Rate Perform the scanning by two or more WAVSs to improve overall Detection Rate
- The evaluation of WAVS is conducted using MusicStore Web Application as a test bed. It is Java based application, which is deployed on Apache Server. The application uses database on Oracle database management server to store the data for the web site in its tables. Because of the widespread use and popularity of those technologies they were chosen as the underlying architecture of the MusicStore. Apache has consistently been the most popular HTTP server since 1995. The latest web server survey conducted in May 2012 found that Apache owns 64.20% of the market share for top servers across all domains. Oracle database is a relational database, which is used extensively all over the world; it is one of the most popular databases around the world. It runs on every platform known, from a mainframe to a Mac. Java is currently one of the most popular programming languages in use, particularly for client-server web applications according to Tiobe. The Java rating is 16.599% that is calculated based on worldwide availability of skilled engineers, courses and third party vendors. The popular search engines like Google, Bing, Yahoo!, Wikipedia, Amazon, YouTube and Baidu are used to calculate its ratings. The decision on using these most popular technologies makes it possible to apply the result of WAVS evaluation to the majority of web applications available currently in the web. The application uses JavaServer Pages (JSP) to present the user interface. It also uses HyperText Markup Language (HTML), Cascading Style Sheets (CSS), JavaScript, and Asynchronous JavaScript and XML (AJAX) technologies.