The Trade-Offs of Transitioning to Open Government Data
1. Transitioning to Open Government Data:
What Are the Trade-Offs?
a federal spending data case study
Becky Sweger
Director of Data and Technology
National Priorities Project
6. Responsibility Shifts to Consumer
“change in emphasis from providing government
information to information consumers to
providing information to mediators…While
information consumers typically require some
analysis and simplification, information
mediators can achieve more innovative
solutions with the most raw form of government
data.”
2nd Principle of Open Government Data from Open Government Data by Josh Tauberer
8. Unavoidable Trade-Offs
Improvement Trade-Off
Timeliness Accuracy
More Data Fields Simplicity & Usability
Granularity Single Format
Granularity Convenient Aggregates
14. Can We Mitigate Trade-Offs & Losses?
• How can we support data consumers without
tech resources?
• What’s a good model for promoting dataset
literacy and community?
• Can open gov types contribute by filling in
some of the gaps?
Notes de l'éditeur
IntroNational Priorities Project: non-profit, non-partisan federal budget research org in Northampton, MA15 years of data-driven projects, started working with federal spending data in early 2011
Why CFFR & USASpending.gov?Both represent federal government spending data – some of the most critical data our government generates and publishesAssistance (grants, loans, payments, insurance)Procurement (contracts)Both derived via same-ish data sources (though CFFR is more inclusive)CFFR: Federal Assistance Awards Data System (FAADS) + Federal Procurement Data System + Salaries + non-FAADS awardsUSASpending.gov: FAADS plus + Federal Procurement Data SystemThese similarities make an interesting case study to compare old and new ways of delivering government dataOut of scope: USASpending.gov data quality. For more information, see http://clearspending.org from the Sunlight Foundation
CFFR – Functional but not sexyAuthorized in 1983Limited interactive queriesBulk downloadsExtensive documentation: methodology, data dictionary, files for supporting codes100+ page PDF with maps, charts, and analysisArchived dataPublished annuallyAggregated by federal program/agency and geographyCFFRCensus BureauAuthorized in 1982 and reported federal spending data from FY 1983 – FY 2010 (http://www.gpo.gov/fdsys/pkg/USCODE-2010-title31/html/USCODE-2010-title31-subtitleV-chap62-sec6202.htm)2 reports: total funds obligated by geography and total funds actually spent by geographyLaw requires 10 data sources in addition to FAADS and FPDSReports/data due 180 days after close of fiscal year“This program provides for the production of the annual Consolidated Federal Funds Report (CFFR) products, including the CFFR Report, on-line query tool, and Federal Aid to States Report. The CFFR products provide statistics on Federal Government expenditures or obligations in each state, territory, county, and municipality. Coverage of the CFFR includes Federal salaries and wages, procurement, grants, and direct payments, as well as Federal insurance and loan programs. The Congress is especially interested in CFFR data as it illustrates the effects of Federal programs on state and sub-state economies. “ Canceled when the Census terminated the Federal Financial Statistic program in the FY 2012 budget in order to “fund higher priority programs within the Census Bureau and Department.” (http://www.osec.doc.gov/bmi/budget/12CJ/Census_Bureau_FY_2012_Congressional_Submission.pdf). Also see the “Strengthening Federal Statistics” section of the FY 2012 Budget Request’s Analytical Perspectives document: http://www.gpo.gov/fdsys/pkg/BUDGET-2012-PER/pdf/BUDGET-2012-PER.pdf (page 342)
OMBFederal Funding Accountability and Transparency Act of 2006 required establishment of a single, searchable website before Jan. 2008 (http://www.gpo.gov/fdsys/pkg/PLAW-109publ282/pdf/PLAW-109publ282.pdf)Data for fiscal year 2007 and each fiscal year thereafterAwards > $25,000Name of entity receiving awardAmount of awardAward info including funding agency, transaction type, program source, award description, NAICS or CFDA code, location of award recipient and place of award performanceGrants AND sub-grantsContracts AND sub-contractsUpdated no later than 30 days after award is granted
USASpending.gov: much more modern websiteWhat we gain from USASpending.govGranularity/flexibilityTimelinessAPIDetailed custom searchInteractive graphs/mapsSub-awards
Shifting the burden to the end user—we need to talk about that.
With more possibilities, comes more work.It’s easier to present accurate data when it’s only scrubbed and presented once a year. Frequent updates to USASpending.gov mean that bad data will find its way in. Of course there’s a process for providing corrections, but the user must know how it works.More data, more visualizations, and a robust custom search = a website that is harder to user and information that is harder to interpret. Consumer will have to make more decisions (e.g., aggregate by place of performance or recipient location?)Transaction-level source records from different systems (awards and procurement) = different download formatsDifferent types of awards records are populated differently
Unreasonable to expect duplicate data entry foreverHowever, it’s unfortunate to have a gap in our spending data, especially at this time
Data dictionary not enforcedZeroes versus blanks versus nullsGeographic inconsistency: for example, two same addresses might have a slightly different zip code
USASpending will not accept support calls, only e-mailSite does list the Senior Accountable Official for each agency, but there is no contact informationhttp://usaspending.gov/learn?tab=Timeliness%20and%20AccountabilityOpen questions about this….How can consumers and the govt. work together to make sure people understand what’s being published?What effect does outsourcing have on the quality of support?
In some ways, the switch from published statistical data to open data makes access less democratic—those without tech resources can’t get what they need.