Professor Martha Bailey, doctoral student Connor Cole, and Alum Morgan Henderson recently published the National Bureau of Economic Research (NBER) working paper entitled, “How Well Do Automated Methods Perform in Historical Samples? Evidence from New Ground Truth.”  In this paper, the authors evaluate the performance of commonly used automated record-linking algorithms in three high quality historical U.S. samples.

 

ABSTRACT:

New large-scale data linking projects are revolutionizing empirical social science. Outside of selected samples and tightly restricted data enclaves, little is known about the quality of these “big data” or how the methods used to create them shape inferences. This paper evaluates the performance of commonly used automated record-linking algorithms in three high quality historical U.S. samples. Our findings show that (1) no method (including hand linking) consistently produces samples representative of the linkable population; (2) automated linking tends to produce very high rates of false matches, averaging around one third of links across datasets and methods; and (3) false links are systematically (though differently) related to baseline sample characteristics. A final exercise demonstrates the importance of these findings for inferences using linked data. For a common set of records, we show that algorithm assumptions can attenuate estimates of intergenerational income elasticities by almost 50 percent. Although differences in these findings across samples and methods caution against the generalizability of specific error rates, common patterns across multiple datasets offer broad lessons for improving current linking practice.