Data Science view of the KDD 2014
•Télécharger en tant que PPTX, PDF•
24 j'aime•15,219 vues
KDD is a premier conference that brings together researchers and practitioners from data science, data mining, knowledge discovery, large-scale data analytics, and big data. These slides give some statistics about the KDD program and present data science view of the paper review process: 1100 submissions, 3000 reviews, and 150 accepted papers.
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (11)
Similaire à Data Science view of the KDD 2014
Similaire à Data Science view of the KDD 2014 (20)
Dernier
Dernier (20)
Data Science view of the KDD 2014
- 1. Data Science of the KDD ‘14 Review Process Jure Leskovec (Stanford) and Wei Wang (UCLA) Joint work with Jason Hirshman and David Zeng (Stanford)
- 2. KDD 2014 Research Track Statistics
- 3. KDD 2014 Program Largest KDD program ever: • 151 research papers (20% growth over KDD’13) • 43 industry & govt. papers (30% growth) • 26 workshops (75% growth) • 11 tutorials (83% growth) Program highlights: • Paper spotlights early morning (8:15am) • Oral presentations (Mon-Wed) • Posters at the reception (Tue night)
- 4. KDD 2014 Research Track • 1036 submissions from 2600 authors – 42% increase over KDD ’13 • 151 papers: – Acceptance rate 14.6% 0 200 400 600 800 1000 1200 2000 2005 2010 2015 KDD year Numberofsubmissions
- 5. KDD Reviewing Process 46 Senior PC members + 340 PC members • 2971 reviews in total (Rough) Acceptance rule: • Raw review score AND Standardized review score AND Raw meta-review AND Standardized meta-review score ≥ Weak Accept • 110 papers matched (immediate accepts) • Remaining papers were discussed with meta-reviewers and final decisions were made
- 7. Acceptance Rate per Country
- 8. Acceptance by Subject Area
- 9. Predicting Paper Acceptance Features Used Accuracy Random Guessing 0.50 Paper Abstract 0.57 Author Status (Past paper counts) 0.64 Author Status (DBLP graph connectivity) 0.61 Author Status (Counts + Graph) 0.65 Reviewer (Similarity, Graph distance to authors) 0.60 All (Abstract, Author Status, and Reviewer) 0.65
- 10. Predicting Paper Acceptance from the Review Text Features Used Paper: Accepted? Review: Score > 0? Random Guessing 0.50 0.50 Review Text 0.68 0.72 Review Text + Numeric Score (Novelty, Presentation) 0.77 0.77 Human Reading of Review Text 0.88 0.73
- 11. I’m submitting a paper: What correlates with acceptance?
- 12. Academia + Industry Papers do Better
- 13. Submissions per Author: 5 is best!
- 14. No benefit in submitting >5 papers!
- 15. Having more authors (seems to) help
- 16. It is the most experienced author that matters!
- 17. What insights can we gain on the review process?
- 18. Most reviews are Weak Rejects
- 19. More granularity is needed at the Weak Reject / Weak Accept level Reviewagreeswiththefinaloutcome
- 20. Review length is a good determinant of a review’s influence/quality Reviewagreeswiththefinaloutcome
- 21. Shorter reviews are used for clear accepts and rejects
- 22. Never review co-author’s papers
- 23. The Curse of the Review Submission Deadline
- 24. Over 50% reviews submitted in the last 5 days Over 20% reviews submitted in the last 24 hours 10% of reviews submitted late
- 25. Ratings increase near the deadline Weak Rejects increase while Rejects decrease
- 26. Reviews submitted late are less likely to agree with final outcome
- 27. Late reviews are shorter
- 28. Review quality drops: Accuracy of predicting score from review text
- 29. Conclusions • To get your papers accepted to KDD: – Collaborate in multidisciplinary teams – Have a senior author on board – Do not submit more than 5 papers • To improve KDD community standards: – Avoid Weak Reject/Weak Accept scores – Write longer and clearer reviews – Submit reviews early!
Notes de l'éditeur
- Country of the paper is given by the mode author nationality. Only countries with more than 10 submissions are shown (except South Korea, which had 0 acceptance)
- Subject areas were based on a field that authors tagged their papers with. Only subject areas with more than 50 submissions are shown.
- On balanced dataset (we subsampled negative examples)
- On balanced dataset (we subsampled negative examples)
- Academic: All authors of paper are affiliated with a university Industry: All authors of paper are affiliated with industry Mixed: Paper has authors affiliated with both universities and industry.