Types of accuracy in satellite image classification are discussed and steps to calculate accuracy

1. ACCURACY ASSESSMENTACCURACY ASSESSMENT

2. ACCURACY ASSESSMENT
– Assess accuracy of a remote sensing output is
one of the most important steps in any
classification exercise!!
– Without an accuracy assessment the output or
results is of little value

3. Goals:
Assess how well a classification worked
Understand how to interpret the usefulness of
someone else’s classification

4. Nature of Classification
1) Class definition problems
2) Inappropriate class labels
3) Mislabeling of classes

5. Accuracy Assessment
 Overview
 Collect reference data: “ground truth”
Determination of class types at specific locations
 Compare reference to classified map
Does class type on classified map = class type
determined from reference data?

6. Reference Data
Some possible sources
Aerial photo interpretation
Ground truth with GPS
GIS layers

7. Reference Data
Issue 1: Choosing reference source
Make sure you can actually extract from the reference
source the information that you need for the
classification scheme
I.e. Aerial photos may not be good reference data if your
classification scheme distinguishes four species of grass. You
may need GPS’d ground data.

8. Reference data
Issue 2: Determining size of reference plots
Match spatial scale of reference plots and
remotely-sensed data
I.e. GPS’d ground plots 5 meters on a side may not be
useful if remotely-sensed cells are 1km on a side. You
may need aerial photos or even other satellite images.

9. Sampling Methods
Simple Random Sampling:
observations are randomly placed.
Stratified Random Sampling
minimum number of observations
are randomly placed in each
category.

10. Sampling Methods
Systematic Sampling
Observations are placed at equal intervals
according to a strategy.
Systematic Non-Aligned Sampling:
a grid provides even distribution of
randomly placed observations.

11. Sampling Methods
Cluster SamplingCluster Sampling
Randomly
placed “centroids” used as a base
of several nearby observations.
The nearby observations can be
randomly selected, systematically
selected, etc...

12. Error matrix
 Summarize using an error matrix
Class types determined from
reference sourcereference source
Class types
determined
from
classified
image
# Plots Cotton Sugarcane Fodder Totals
Cotton 50 5 2 57
Sugarcane 14 13 0 27
Fodder 3 5 8 16
Totals 67 23 10 100

13. Total Accuracy
 Quantifying accuracy
– Total Accuracy: Number of correct plots / total number of
plots
Class types determined from
reference sourcereference source
Class
types
determin
ed from
classifie
d map
# Plots Cotton Sugarcane Fodder Totals
Cotton 50 5 2 57
Sugarcane 14 13 0 27
Fodder 3 5 8 16
Totals 67 23 10 100
%71100*
100
81350
=
++
=AccuracyTotal
Diagonals represent
sites classified correctly
according to reference
data
Off-diagonals were mis-
classified

14. Total Accuracy
Problem with Total Accuracy:
Summary value is an average
Does not reveal if error was evenly distributed between
classes or if some classes were really bad and some
really good
Therefore, include other forms:
User’s accuracy
Producer’s accuracy

15. User’s and producer’s accuracy
and types of error
User’s accuracy corresponds to error of
commission (inclusion):
Producer’s accuracy corresponds to error of
omission (exclusion):

16. User’s Accuracy
From the perspective of the user of the
classified map, how accurate is the map?
 For a given class, how many of the pixels on the
map are actually what they say they are?
– Calculated as:
Number correctly identified in a given map class /
Number claimed to be in that map class

17. User’s Accuracy
Class types determined from
reference sourcereference source
Class
types
determin
ed from
classifie
d map
# Plots Cotton Sugarcane Fodder Totals
Cotton 50 5 2 57
Sugarcane 14 13 0 27
Fodder 3 5 8 16
Totals 67 23 10 100
%88100*
57
50
,'
==Accuracy CottonsUser
Example: Cotton

18. Producer’s Accuracy
From the perspective of the maker of the
classified map, how accurate is the map?
 For a given class in reference plots, how many of
the pixels on the map are labeled correctly?
– Calculated as:
Number correctly identified in ref. plots of a given
class /
Number actually in that reference class

19. Producer’s Accuracy
Class types determined from
reference sourcereference source
Class
types
determine
d from
classified
image
# Plots Conifer sugarcane Fodder Totals
Cotton 50 5 2 57
sugarcane 14 13 0 27
Fodder 3 5 8 16
Totals 67 23 10 100
%75100*
67
50
'
==Accuracy s,Cottonproducer
Example: Cotton

20. Summary so far
Class types determined from
reference sourcereference source
User’s
AccuracyClass types
determined from
classified
Image
# Plots Cotton Sugarcane Fodder Totals
Cotton 50 5 2 57 88%
Sugarcane 14 13 0 27 48%
Fodder 3 5 8 16 50%
Totals 67 23 10 100
Producer’s Accuracy 75% 57% 80% Total: 71%

21. Accuracy Assessment in ERDAS
Imagine
 Make a file in Excel with ONLY the X-Y
coordinates for your GPS points collected in field

22.  Now save this new file as a .txt file (Tab delimited)

23.  In ERDAS From the Accuracy Assessment window
go to Edit > Import User-defined Points.

24.  Put GPS co-ordinates file and click ok the window
opens as:

25.  Now you can enter your class values for the
reference numbers in the Accuracy Assessment table

26.  Go to Edit > Show Class Values and the Class
column will be filled in with the values that are the actual
values of the pixels located at the X-Y coordinates that
you indicated

27.  Finally, you can look at the statistics of your accuracy
assessment by first going to Report > Options. Then
go to Report > Accuracy Report and the following table
will appear

28. References
• Lillesand and Kiefer, Chapter 7
• Congalton, R. G. and K. Green. 1999. Assessing the
accuracy of remotely sensed data: Principles and
practices. Lewis Publishers, Boca Raton.
• Congalton, R.G. 1991. A review of assessing the
accuracy of classification of remotely sensed data.
Remote Sensing of Environment 37:35-46
• ERDAS, (2005). ERDAS 9.1, Field Guide. Geospatial
Imaging, LLC Norcross, Georgia.
• Van Neil, T.G. and McVicar, T.R., (2004) Current and
potential uses of optical remote sensing in rice based
irrigation systems: A review, Australian Journal of
Agricultural Research, 55, 155-85, CSIRO