Presented in the Biodiversity Informatics Services & Workflows symposium at the 2015 TDWG meeting.

1. Strings to things: a user-friendly
framework for data reconciliation
Nicky Nicolson, RBG Kew
@nickynicolson
Biodiversity Information Standards (TDWG) annual meeting
Nairobi, Kenya / 28th September – 1 October 2015

2. Reconciliation
• Turns a string representation of an entity into
an actionable identifier.
e.g.:
Tahina spectabilis
Will reconcile to:
http://ipni.org/urn:lsid:ipni.org:names:77086615-1

4. Maximise reuse, two stage process
1. Standardise data
- Package of 40 plus “transformers”
- All accept a string input, produce a string
output

5. Examples of transformers

6. Open Refine screenshot

7. Open source
http://github.com/RBGKew/StringTransformers

8. Maximise reuse, two stage process
2. Match the data
- Package of 20 plus “matchers”
- All accept two inputs and return a flag if they
match

9. Configuring a service
1) Read tabular data (file or DB)
2) Configure transformers
3) Configure matchers

10. Run it…
1) Service description
2) Three service endpoints
3) Javascript query interface

11. IPNI Reconciliation Service

12. 3 service endpoints

13. IPNI Reconciliation Service

14. Flexible web service
• Open Refine compatible
• But underneath it’s JSON over HTTP
• … so call it from any programming language

15. Service metadata

16. Service call

17. Service response

18. List of reconciliation services
https://github.com/OpenRefine/OpenRefine/wi
ki/Reconcilable-Data-Sources

19. Open source
https://github.com/RBGKew/Reconciliation-and-Matching-Framework

20. What we’ll work on in the future

21. Reconciliation services on different
data types
• Specimens
– Add DwCA as a readable data store
– Collections focussed transformers & matchers
– Resolve & link specimen duplicates
• People
• Trait glossaries

22. Integration with github

24. Thanks to:
• Biodiversity Informatics team (Abigail Barker,
Matt Blissett, James Crowe, John Iacona, Rob
Turner, Alecs Gueder)
• Plant & fungal name curation team (Christine
Barker / Irina Belyaeva / Katherine Challis /
Rafael Govaerts / Paul Kirk / Heather Lindon /
Emma Williams)
• Data improvement team (Anna Lynch, Rachel
Witherow, Malin Rivers, Esther Wainwright-Deri)

25. @nickynicolson / n.nicolson@kew.org
http://bit.ly/k-names-service
http://github.com/RBGKew
Biodiversity Information Standards (TDWG) annual meeting
Nairobi, Kenya / 28th September – 1 October 2015