1. Unit 4 Study Guide
Chapter 12: Agency Records, Content Analysis, and Secondary Data
Learning Objectives:
1. Recognize that public organizations produce statistics and data that are often useful for
criminal justice researchers.
2. Provide examples of nonpublic agency records that can serve as data for criminal justice
research.
3. Understand why the units of analysis represented by agency data may be confusing for
researchers.
4. Explain why researchers must be attentive to reliability and validity problems that might
stem from agency records.
5. Summarize why “follow the paper trail” and “expect the expected” are useful maxims to
follow when using agency records in research.
6. Summarize content analysis as a research method appropriate for studying
communications.
7. Describe examples of coding to transform raw data into a standardized, quantitative form.
8. Summarize how secondary analysis refers to the analysis of data collected by another
researcher for some other purpose.
9. Be able to access archives of criminal justice data that are maintained by the ICPSR and
the NACJD.
10. Understand how the advantages and disadvantages of secondary data are similar to those
for agency records.
Chapter Summary:
• Many public organizations produce statistics and data for the public record, and these data are
often useful for criminal justice researchers.
• All organizations keep nonpublic records for internal operational purposes, and these records
are valuable sources of data for criminal justice research.
• Public organizations can sometimes be enlisted to collect new data-- through observations or
interviews-- for use by researchers.
• The units of analysis represented by agency data may not always be obvious, because agencies
typically use different, and often unclear, units of count to record information about people and
cases.
• Researchers must be especially attentive to possible reliability and validity problems when they
use data from agency records.
• “Follow the paper trail” and “Expect the expected” are two general maxims for researchers to
keep in mind when using agency records in their research.
• Content analysis is a research method appropriate for studying human communications.
Because communication takes many forms, content analysis can study many other aspects of
behavior.
• Coding is the process of transforming raw data-- either manifest or latent content-- into a
standardized, quantitative form.
2. • Secondary analysis is the analysis of data collected earlier by another researcher for some
purpose other than the topic of the current study.
• Archives of criminal justice and other social data are maintained by the ICPSR and the NACJD
for use by other researchers.
• The advantages and disadvantages of using secondary data are similar to those for agency
records-- data previously collected by another researcher may not match our own needs.
Key Terms:
Content analysis (Page 348)
Interuniversity Consortium for Political and Social Research (ICPSR) (Page 353)
Latent content (Page 349)
Manifest content (Page 348)
National Archive of Criminal Justice Data (NACJD) (Page 353)
Published statistics (Page 331)
Secondary analysis (Page 330)
Social production of data (Page 334)
Chapter 13: Evaluation Research and Problem Analysis
Learning Objectives:
1. Summarize evaluation research and problem analysis as examples of applied research in
criminal justice.
2. Describe how different types of evaluation activities correspond to different stages in the
policy process.
3. Explain the role of an evaluability assessment.
4. Understand why a careful formulation of the problem, relevant measurements, and criteria
of success or failure are essential in evaluation research.
5. Describe the parallels between evaluation research designs and other designs.
6. Explain the advantages, requirements, and limits of randomized field experiments.
7. Summarize the importance of process evaluations conducted independently or in
connection with an impact assessment.
8. Describe the role of problem analysis as a planning technique that draws on the same social
science research methods used in program evaluation.
9. Explain how the scientific realist approach focuses on mechanisms in context, rather than
generalizable causal processes.
10. Present an example of how criminal justice agencies are increasingly using problem
analysis tools, crime mapping, and other space-based procedures.
11. Explain how evaluation research entails special logistical, ethical, and political problems.
Chapter Summary:
• Evaluation research and problem analysis are examples of applied research in criminal justice.
• Different types of evaluation activities correspond to different stages in the policy process--
policy planning, process evaluation, and impact evaluation.
3. • An evaluability assessment may be undertaken as a scouting operation or a preevaluation to
determine whether it is possible to evaluate a particular program.
• A careful formulation of the problem, including relevant measurements and criteria of success
or failure, is essential in evaluation research.
• Organizations may not have clear statements or ideas about program goals. In such cases,
researchers must work with agency staff to formulate mutually acceptable statements of goals
before proceeding.
• Evaluation research may use experimental, quasi-experimental, or nonexperimental designs. As
in studies with other research purposes, designs that offer the greatest control over
experimental conditions are usually preferred.
• Randomized designs cannot be used for evaluations that begin after a new program has been
implemented or for full-coverage programs in which it is not possible to withhold an
experimental treatment from a control group.
• Process evaluations can be undertaken independently or in connection with an impact
assessment. Process evaluations are all but essential for interpreting results from an impact
assessment.
• Problem analysis is more of a planning technique. However, problem analysis draws on the
same social science research methods used in program evaluation. Many variations on problem
analysis are used in applied criminal justice research.
• The scientific realist approach to applied research focuses on mechanisms in context, rather
than generalizable causal processes.
• Criminal justice agencies are increasingly using problem analysis tools for tactical and
strategic planning. Crime mapping and other space-based procedures are especially useful
applied techniques.
• Problem solving, evaluation, and scientific realism have many common elements.
• Evaluation research entails special logistical, ethical, and political problems because it is
embedded in the day-to-day events of public policy and real life.
Key Terms:
Evaluation research (Page 362)
Evidence-based policy (Page 363)
Impact assessment (Page 366)
Problem analysis (Page 362)
Problem-oriented policing (Page 384)
Problem solving (Page 384)
Process evaluation (Page 367)
Stakeholders (Page 370)
Chapter 14: Interpreting Data
Learning Objectives:
1. Understand that descriptive statistics are used to summarize data under study.
2. Describe a frequency distribution in terms of cases, attributes, and variables.
4. 3. Recognize that measures of central tendency summarize data, but they do not convey the
idea of original data.
4. Understand that measures of dispersion give a summary indication of the distribution of
cases around an average value.
5. Provide examples of rates as descriptive statistics that standardize some measure for
comparative purposes.
6. Describe how bivariate analysis and subgroup comparisons examine relationships between
two variables.
7. Compute and interpret percentages in contingency tables.
8. Understand that multivariate analysis examines the relationships among several variables.
9. Explain the logic underlying the proportionate reduction of error (PRE) model.
10. Describe the use of lambda and gamma, and and Pearson’s product-moment correlation as
PRE-based measures of association for nominal, ordinal, and interval/ration variables,
respectively.
11. Summarize how regression equations and regression lines are used in data analysis.
12. Understand how inferential statistics are used to estimate the generalizability of findings
arrived at in the analysis of a sample to a larger population.
13. Describe the meaning of confidence intervals and confidence levels in inferential statistics.
14. Explain what tests of statistical significance indicate, and how to interpret them.
Chapter Summary:
• Descriptive statistics are used to summarize data under study.
• A frequency distribution shows the number of cases that have each of the attributes of a given
variable.
• Measures of central tendency reduce data to an easily manageable form, but they do not
convey the detail of the original data.
• Measures of dispersion give a summary indication of the distribution of cases around an
average value.
• Rates are descriptive statistics that standardize some measure for comparative purposes.
• Bivariate analysis and subgroup comparisons examine some type of relationship between two
variables.
• The rules of thumb in making subgroup comparisons in bivariate percentage tables are (1)
“percentage down” and “compare across” or (2) “percentage across” and “compare down.”
• Multivariate analysis is a method of analyzing the simultaneous relationships among several
variables and may be used to more fully understand the relationship between two variables.
• Many measures of association are based on a proportionate reduction of error (PRE) model,
which measures improvement in predictions about one variable, given information about a
second variable.
• Lambda and gamma are PRE-based measures of association for nominal and ordinal variables,
respectively.
• Pearson’s product-moment correlation is a measure of association used in the analysis of two
interval or ratio variables.
• Regression equations are computed based on a regression line-- the geometric line that
represents, with the least amount of discrepancy, the actual location of points in a scattergram.
5. • The equation for a regression line predicts the values of a dependent variable based on values
of one or more independent variables.
• Inferential statistics are used to estimate the generalizability of findings arrived at in the
analysis of a sample to the larger population from which the sample has been selected.
• Inferences about some characteristic of a population, such as the percentage that favors gun
control laws, must contain an indication of a confidence interval (the range within which the
value is expected to be-- for example, between 45 and 55 percent favor gun control) and an
indication of the confidence level (the likelihood that the value does fall within that range-- for
example, 95 percent confidence).
• Tests of statistical significance estimate the likelihood that an association as large as the
observed one could result from normal sampling error if no such association exists between the
variables in the larger population.
• Statistical significance must not be confused with substantive significance, which means that
an observed association is strong, important, or meaningful.
• Tests of statistical significance, strictly speaking, make assumptions about data and methods
that are almost never satisfied completely by real social research. Claiming a “statistically
discernible relationship” is more appropriate when assumptions are not satisfied.
Key Terms:
Average (Page 401)
Bivariate analysis (Page 408)
Central tendency (Page 401)
Contingency table (Page 411)
Descriptive statistics (Page 399)
Dispersion (Page 401)
Frequency distributions (Page 400)
Inferential statistics (Page 399)
Level of significance (Page 423)
Mean (Page 401)
Median (Page 401)
Mode (Page 401)
Multivariate analysis (Page 411)
Nonsampling error (Page 422)
Null hypothesis (Page 425)
Proportionate reduction or error (PRE) (Page 417)
Range (Page 401)
Regression analysis (Page 419)
Standard deviation (Page 403)
Statistical significance (Page 423)
Statistically discernible difference (Page 428)
Tests of statistical significance (Page 422)
Univariate analysis (Page 400)
6. • The equation for a regression line predicts the values of a dependent variable based on values
of one or more independent variables.
• Inferential statistics are used to estimate the generalizability of findings arrived at in the
analysis of a sample to the larger population from which the sample has been selected.
• Inferences about some characteristic of a population, such as the percentage that favors gun
control laws, must contain an indication of a confidence interval (the range within which the
value is expected to be-- for example, between 45 and 55 percent favor gun control) and an
indication of the confidence level (the likelihood that the value does fall within that range-- for
example, 95 percent confidence).
• Tests of statistical significance estimate the likelihood that an association as large as the
observed one could result from normal sampling error if no such association exists between the
variables in the larger population.
• Statistical significance must not be confused with substantive significance, which means that
an observed association is strong, important, or meaningful.
• Tests of statistical significance, strictly speaking, make assumptions about data and methods
that are almost never satisfied completely by real social research. Claiming a “statistically
discernible relationship” is more appropriate when assumptions are not satisfied.
Key Terms:
Average (Page 401)
Bivariate analysis (Page 408)
Central tendency (Page 401)
Contingency table (Page 411)
Descriptive statistics (Page 399)
Dispersion (Page 401)
Frequency distributions (Page 400)
Inferential statistics (Page 399)
Level of significance (Page 423)
Mean (Page 401)
Median (Page 401)
Mode (Page 401)
Multivariate analysis (Page 411)
Nonsampling error (Page 422)
Null hypothesis (Page 425)
Proportionate reduction or error (PRE) (Page 417)
Range (Page 401)
Regression analysis (Page 419)
Standard deviation (Page 403)
Statistical significance (Page 423)
Statistically discernible difference (Page 428)
Tests of statistical significance (Page 422)
Univariate analysis (Page 400)
