Presentation slides of MSR 2018 article, co-authored by Alexandre Decan, Tom Mens and Eleni Constantinou from University of Mons, Belgium. Research carried out as part of the SECOHealth and SECO-ASSIST research projects. Abstract: Security vulnerabilities are among the most pressing problems in open source software package libraries. It may take a long time to discover and fix vulnerabilities in packages. In addition, vul- nerabilities may propagate to dependent packages, making them vulnerable too. This paper presents an empirical study of nearly 400 security reports over a 6-year period in the npm dependency network containing over 610k JavaScript packages. Taking into account the severity of vulnerabilities, we analyse how and when these vulnerabilities are discovered and fixed, and to which extent they affect other packages in the packaging ecosystem in presence of dependency constraints. We report our findings and provide guidelines for package maintainers and tool developers to improve the process of dealing with security issues.

1. On the impact of security vulnerabilities in
the npm package dependency network
Alexandre Decan
Tom Mens
Eleni Constantinou
Replication package
https://doi.org/10.5281/zenodo.1193577

2. Motivation: HeartBleed bug
Introduced in 2010
• Allowed anyone on the Internet to read the
memory of the software systems
• Simple programming mistake
Discovered and traced in April 2014
• 0.5M servers certified by trusted authorities
were believed to be a affected

3. Motivation: Security
Vulnerabilities in OSS
• OSS widely used, even in commercial software
• Vulnerabilities often found years after their introduction
• Exploited vulnerabilities may compromise many dependent
applications

4. Motivation: Security Vulnerabilities in OSS
https://www.blackducksoftware.com/technology/vulnerability-reporting
time

5. Focus: Security vulnerabilities in
package dependency networks

6. Focus: Security vulnerabilities in
package dependency networks

7. Focus: Security vulnerabilities in
package dependency networks

8. Most packages depend
on another one.
~60% in April 2016

9. vulnerability dataset
Package metadata gathered
from libraries.io (automatically)
on November 2017
610,097 packages
Packages releases 4,202,099
Runtime package dependencies 20,240,402

10. vulnerability dataset
399 vulnerabilities
Affected packages 269
# releases of affected packages 14,931
# affected releases 6,752
Vulnerabilities gathered
from snyk.io (manually)

11. vulnerability dataset
399 vulnerabilities
Affected packages 269
# releases of affected packages 14,931
# affected releases 6,752
Vulnerabilities gathered
from snyk.io (manually)

12. How long do packages
remain vulnerable?
>40% of all vulnerabilities are still there after 2 years,
regardless of their severity.

13. How long do packages
remain vulnerable?
40% of all vulnerabilities are not fixed after 2 years,
regardless of their severity.
It takes a long time before vulnerabilities are
removed from a package.

14. When are vulnerabilities
fixed?
+ Most vulnerabilities are quickly fixed after their discovery.
- ~20% of vulnerabilities take more than 1 year to be fixed.

15. When are vulnerabilities
fixed?
Most vulnerabilities are fixed after the reported discovery
date but before they become public.

16. When are vulnerabilities
fixed?
Vulnerabilities must be fixed early/before
public announcement. Unmaintained
vulnerable packages should be deprecated.

17. Vulnerable packages
# vulnerable packages 269
# releases of vulnerable packages 14,931
# vulnerable releases 6,752
# dependent packages affected
by the vulnerable packages
72,470
To which extent do vulnerabilities
affect dependent packages?

18. Package maintainers must use security
monitoring tools, and adapt their dependency
constraints to automatically quickly benefit
from security fixes
To which extent do vulnerabilities
affect dependent packages?

19. A large fraction of affected dependent packages are not updated,
even if an upstream fix is available.
Status of affected dependents that are not yet fixed
To which extent do vulnerabilities
affect dependent packages?

20. Status of affected dependents that are not yet fixed
To which extent do vulnerabilities
affect dependent packages?
Main causes of not fixing a vulnerability in a dependent package:
• Improper or too restrictive use of dependency constraints
• Dependent package is no longer actively maintained

21. Only about half of all dependents are fixed before or at same time as upstream fix.
>33% of all affected dependents are not (yet) fixed!
To which extent do vulnerabilities
affect dependent packages?

22. Main causes of not fixing a vulnerability in a dependent package:
• Improper or too restrictive use of dependency constraints
• Dependent package is no longer actively maintained
To which extent do vulnerabilities
affect dependent packages?

23. Summary
• Vulnerabilities linger for a long time
until they are discovered
• One of of five security vulnerabilities
take more than 1 year to fix after its
discovery
• Many dependent packages do not
upgrade to security fixes in upstream
packages.

24. Solutions
• Better use of dependency
constraints and semantic
versioning
• Use security monitoring tools
• Deprecate unmaintained
obsolete packages
• Use better versioning and
security policies