This document describes a system for recognizing easily confused traditional Chinese medicine (TCM) herbs using convolutional neural networks (CNNs) on smartphones. The system was trained on a dataset of 2400 herb images from 24 categories collected using smartphone cameras. A hierarchical clustering CNN approach achieved higher accuracy than a standard CNN, with accuracy over 97% for most herb categories. Testing the system on different smartphone models found varying accuracy rates due to differences in camera quality. Data augmentation techniques like image rotation and brightness adjustment were also found to improve classification accuracy over using a single phone's dataset.
2. A little about me
• Background in sensor network (aka. IoT)
•2011: experienced TCM
•2013: started doing research on TCM
• smartphone APPs for TCM
• Tongue diagnosis (https://lens.csie.ncku.edu.tw/~john/)
• AR-based acupoint localization
(https://www.youtube.com/watch?time_continue=1&v=RyzKMuo3Gjo)
• TCM Herb recognition
•2015: studying TCM at China Medical University
(中國醫藥⼤學)
2
GTC Taiwan 2017
3. TCM 101
• Based on thousands of years of clinical experiences
• Data -> model (similar to DNN?)
• Treat by symptom 症(personalized treatment)
• Considering individual constitution and the interaction with the
environment
• Western Medicine : Treat by disease 病(same treatment for same disease)
• Four diagnoses (四診) : collect biometrics using sensors on the human body
• Inspection (望)
• Listen and smell (聞)
• Inquiry (問)
• Palpation (切)
3
GTC Taiwan 2017
30. Future work
• Short term
• Collect data for all 300+ TCM
herbs
• Try with more different phones
under different lighting conditions
• Long term
• A TCM robot assistant
30
GTC Taiwan 2017