A caution on using charity financial data
Posted under News | Canadian Charity Statistics
The US National Centre for Charitable Statistics has some good cautions on the use of US data Form 990 data that I recently noted. Similar issues apply here in Canada with the T3010 so I have reproduced the sections below.
Limitations of the Data and Recommended Error Checking Procedures
Financial data in the nonprofit database files should be used with caution. While NCCS performs some data checks, errors still exist. We strongly advise checking outliers and finances of large individual organizations before publishing analyses of the data sets. In addition to the potential data-entry errors discussed above, researchers have questioned the reliability of the underlying data on the Form 990, Form 990-EZ, or Form 990-PF. Preparers, especially in small nonprofit organizations, may not fully understand the complexities of the financial entries. Organizations may also shift expenses from one category to another to obtain desired ratios. While studies have shown aggregate measures to be reliable, financial entries requiring exclusions or multiple calculations are less so. (For a more detailed discussion, see Froelich, Knoepfle, and Pollak 2000).
A four-stage error checking process is recommended. When using financial data, it is important to keep in mind that there is an enormous range in size between the smallest and the largest nonprofit organizations. A single hospital or university in a small state may account for more than 20 percent of the nonprofit revenues or assets within the state. Thus, if data on the nonprofit sector are being aggregated, the finances of any dominant organizations should be reviewed.
First, the researcher should identify large organizations that dominate analytic categories. When studying the sector as a whole, any inaccuracy, accounting change, or anomaly in the financial reports of a single large hospital, for instance, can mask the financial trends of thousands of small human service organizations. Such misleading data may also be found within individual subsectors. For example, one large arts organization in New England accounts for a substantial percentage of the income for the region’s nonprofit arts community. When the way it accounted for some income was changed from one year to the next, the aggregate data then seemingly indicated a “trend” for the entire arts and culture subsector in New England.
Second, geographic information and NTEE classifications should be verified for accuracy (or, at minimum, plausibility). On relatively rare occasions, a parent organization will file returns for multiple affiliates. These affiliates may use the legal address of the parent organization, thus inflating the number of organizations and nonprofit activity in a particular city. (Jackson, MS, Los Angeles, CA, Pittsburgh, PA, and Missoula, MT are four known cases where this has occurred.) NCCS has verified the consistency of zip codes and states in many of its major files. However, it is still possible that some cases have eluded us. (Some Core files, for example, might contain incorrect state abbreviations.) As of Fall 2006, the NCCS team is in the process of fixing the remainder of the files that have inconsistencies between state and zip code values. As discussed later in this guide, NTEE codes should also be checked carefully for organizations that have a large impact on your analysis. NCCS can provide researchers with online tools for facilitating this review.
Third, financial data outliers (if they are likely to meaningfully affect your analysis) and “suspicious” dominant organizations should be checked, one return at a time. Perusal of a Form 990 (and its attachments) on GuideStar or the Foundation Center may be sufficient. In other cases, it is often most efficient to call an organization directly and ask for clarification about its Form 990 data. Another approach is to obtain financial or programmatic information from other sources. Data on higher education institutions and hospitals, for example, may be acquired from some states, the Centers for Medicare & Medicaid Services, or private sources. However, these data, in our experience, are often more difficult to match and compare than
researchers initially expect, especially if they lack EINs. (NCCS can provide sophisticated name and address matching programs to help in this case.) Some state charity offices also make their data available.
Fourth, adjustments should be made to the data where appropriate. If accounting changes or corporate transactions (mergers, spin-offs, etc.) are the cause of financial anomalies, one may choose to impute financial measures to account for the effect of the transactions. If simple errors in data entry are found, they should be corrected. In order to improve the quality of our data, NCCS would appreciate feedback (ideally in electronic format) on errors identified and adjustments made so that our master files can be updated for future research.”