Contemporary philippine arts from the regions_PPT_Module_12 [Autosaved] (1).pptx
Do we really need large spectral libraries for the assessment of soil organic carbon at local scale?
1. Do we really need large spectral libraries for the
assessment of soil organic carbon at local scale?
1 University Miguel Hernandez of Elche (Spain); 2 SLU (Sweden); 3 University Rey Juan Carlos (Spain); 4
CSIRO – Land & Water (Australia); 5 Cranfield University (UK); 6 University Polytechnic Cartagena (Spain);
7 University of Málaga (Spain).
César Guerrero1, Johanna Wetterlind2, Bo Stenberg2, Fernando T. Maestre3,
Raphael A. Viscarra Rossel4, Abdul M. Mouazen5, Boyan Kuang5, Raúl Zornoza6, J.D. Ruiz-Sinoga7
2. Introduction
* Many times, we need to analyze a large number of samples collected at local
scale (i.e., collected in a target site).
* Options and approaches for local-scale assessment with NIR spectroscopy:
i) LOCAL models (spectrum-specific model)
• Models constructed with similar spectra
• Expensive: a large library is needed for select enough similar spectra
ii) Geographically local model (site-specific model)
• Models constructed with similar samples (similar matrix)
• Expensive approach: these models are only valid for the target site
iii) Large-scale models (regional, national, continental…)
• A large model is not a guarantee of accurate predictions (large variability)
• Prone to include non-linear patterns: biased predictions
• We can apply “spiking”
3. Introduction
SOC NIR spectra
Y
X
SOC NIR spectra
Y
X
Kennard-Stone
algorithm
Spiked Calibration
Spectral
Library
Target
Site
Prediction
Set
Calibration
R2, RMSEP, bias,
SEP, RPD
Spectral
Library
Target
Site
Prediction
Set
ŷ
R2, RMSEP, bias,
SEP, RPD
ŷ
Spiking
Subset
PLS-R PLS-R
Target site
(unknown)
Target site
(unknown)
SOC ? SOC ?
Spiking
- Useful approach when large-scale models (regional or national) are used at local scale (in a target site).
- This approach ensures that model contains similar samples (to those to be predicted).
- Implies analytical efforts: the spiking subset must be analyzed with the reference method.
- The spiking subset should be small: small contribution on the model (especially if the calibration is big).
PLS-RPLS-R
Unspiked Spiking
X X
Spiking subset
6. 0
2
4
6
8
10
0 2 4 6 8 10
Spiking subset extra-weighted
SOC NIR spectra
Y X
Copy
Paste
Paste
Paste
Paste
copies
Library samples
Spiking subset
cross-validation
the calibration is forced to fit better the spiking subset
Spiking
cross-validation
0
2
4
6
8
10
0 2 4 6 8 10
Spiking + extra-weighting
Spiked Calibration with
extra-weighting
Spectral
library Target
Site
Prediction
Set
Copies
Kennard -Stone
algorithm
7. SOC NIR spectra
Y
X
SOC NIR spectra
Y
X
Target site
(unknown)
Target site
(unknown)
SOC ?
SOC ?
PLS-R
PLS-R
X
X
Spiking subset (n=8)
SOC NIR spectra
Y
X Target site
(unknown)
SOC ?
PLS-R
X
Spiking subset (n=8)
Copies = Extra weight
Unspiked
calibration
Spiked
calibration
Spiked calibration
with Extra-weight
Unspiked
Calibration
Spectral
library
Target
Site
Prediction
Set
Spiked
Calibration
Spectral
library
Target
Site
Prediction
Set
Spiked Calibration
with extra-weighting
Spectral
library Target
Site
Prediction
Set
Copies
Calibration types
8. Unspiked Spiking Spiking & Extra-weighting
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
9. Unspiked
National set Spain
n=1096
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
10. Unspiked
National set Spain
n=1096
Sweden
n=125
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
11. Unspiked
National set Spain
n=1096
Valencia
n=121
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
12. Unspiked
National set Spain
n=1096
Forest, Alicante
n=130
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
13. Unspiked
National set Spain
n=1096
UK
n=104
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
14. Unspiked
National set Spain
n=1096
Forest, Murcia
n=156
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
15. Unspiked
National set Spain
n=1096
Gypsipherous
n=95
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
16. Unspiked
National set Spain
n=1096
Almería (mica schists)
n=60
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
17. Unspiked
National set Spain
n=1096
Cuenca
n=55
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
18. Unspiked
National set Spain
n=1096
Palencia
n=55
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
19. Unspiked
National set Spain
n=1096
Total
n=901
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
R2, RMSEP, bias,
SEP, RPIQ, RPD
20. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
21. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
22. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Sweden
n=125
23. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Valencia
n=121
24. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Forest, Alicante
n=130
25. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
UK
n=104
26. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Forest, Murcia
n=156
27. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Gypsipherous
n=95
28. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Almería (mica schists)
n=60
29. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Cuenca
n=55
30. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Palencia
n=55
31. National set Spain
n=1096
Total
n=901
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
R2, RMSEP, bias,
SEP, RPIQ, RPD
32. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
33. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
34. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Sweden
n=125
35. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Valencia
n=121
36. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Forest, Alicante
n=130
37. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
UK
n=104
38. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Forest, Murcia
n=156
39. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Gypsipherous
n=95
40. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Almería (mica schists)
n=60
41. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Cuenca
n=55
42. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
Palencia
n=55
43. National set Spain
n=1096
Total
n=901
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
R2, RMSEP, bias,
SEP, RPIQ, RPD
44. National set Spain
n=1096
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
45. Predictions: national set (n=1096)
0.85
0.87
0.96
Unspiked Spiked Spiked EW
2.59
2.22
0.71
Unspiked Spiked Spiked EW
1.34
1.56
4.89
Unspiked Spiked Spiked EW
1.65
1.23
0.21
Unspiked Spiked Spiked EW
R2 RMSEP (%SOC) |Bias| (%SOC) RPIQ
Measured SOC (%)
0 5 10 15 20
PredictedSOC(%)
0
5
10
15
20
Unspiked Spiked Spiked & extra-weight
n=901
49. 33 % National set Spain
n=362
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
50. 1.56
0.95
0.20
Unspiked Spiked Spiked EW
Measured SOC (%)
0 5 10 15 20
PredictedSOC(%)
0
5
10
15
20
Predictions: 33% national set (n=362)
R2 RMSEP (%SOC) RPIQ
0.85
0.90
0.96
Unspiked Spiked Spiked EW
2.57
1.85
0.71
Unspiked Spiked Spiked EW
1.35
1.87
4.90
Unspiked Spiked Spiked EW
Unspiked Spiked Spiked & extra-weight
n=901
|Bias| (%SOC)
54. Provincial set (Alicante)
n=147
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
59. Geograph. local set (Alicante)
n=88
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
60. 1.80
0.06 0.01
Unspiked Spiked Spiked EW
Predictions: Geographical local set (n=88)
R2 RMSEP (%SOC) RPIQ
0.17
0.96 0.96
Unspiked Spiked Spiked EW
4.45
0.65 0.60
Unspiked Spiked Spiked EW
0.78
5.34
5.75
Unspiked Spiked Spiked EW
Measured SOC (%)
0 5 10 15 20
PredictedSOC(%)
0
5
10
15
20
Unspiked Spiked Spiked & extra-weight
n=901
|Bias| (%SOC)
64. 50% Provincial set
n=73
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
69. Geograph. local set #2
(Guadalajara)
n=40
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
74. 25% Provincial set
n=36
Unspiked
Unspiked Calibration
(UC)
Spectral
library
Target
Site
Prediction
Set
Spiking Spiking & Extra-weighting
Spiked Calibration
(SC)
Spectral
library Target
Site
Prediction
Set
Spiked Calibration with
extra-weighting (SC+EW)
Spectral
library Target
Site
Prediction
Set
Copies
Material & methods
86. Conclusions and recommendations
• When a large-sized calibration is spiked, the spiking subset should be
extra-weighted. Otherwise, the impact of the spiking subset will be low
(especially if the spiking subset is small).
• For local scale assessment, we don’t need the development of large
spectral libraries: once spiked, the small-sized performed even better than
the large-sized calibrations.
• Do not focus your analytical efforts exclusively in the development of
large-sized libraries and large-sized models. Split your analytical efforts in:
• the development of small spectral libraries (a few regional samples)
• the analysis of the spiking subsets
• This approach (spiking + extra-weight) gives accurate results and small
analytical efforts are needed (new users attracted, more soils would be
studied, more benefits for soil management)
87. Thank you very much for your attention!!
Criticisms welcomed!!
Acknowledgements
This work is part of the research project (Ref. CGL2011-27001) “Estimación de carbono
orgánico en suelos mediante espectroscopía de infrarrojo cercano (NIR): estrategias para
el desarrollo de modelos con aplicabilidad a escala nacional” financed by the Spanish
“Ministerio de Economía y Competitividad” (Plan Nacional I+D+i)