Overview/Narrative

When surveys are taken, the situation can arise where one or more of the persons taking the survey do not actually interview a person, but instead, make up the data. There are a number of reasons why this may happen - e.g. time pressure to complete the results, inconvenient to visit, too far to drive, too late in the day, etc. The industry term for this is "curb stoning", inspired by an image of the surveyor completing the survey while sitting on a curb stone.

It is essential to detect when instances of this may be happening, as the data is made up, it can invalidate the survey study results. One means of detecting potential instances of curb stoning is through the application of Benford's Law, which provides an expected distribution for the frequency of first digits. This law has been found to apply to many numerical results, such as lengths of rivers, number of people residing in a household, etc.

Audit Objective

The objective is to identify potential instances of curb stoning based upon a comparison of the survey data completed compared with that which would be expected, if the data distribution conformed with Benford's Law. Note that some elements of a survey will NOT comply with Benford's law where they are of a predetermined range. However, in this case, all the data elements should conform - i.e. street address number, number of people in the household, and the distance to the nearest church.

Audit Procedures / Audit Program

1. Obtain survey data in electronic format, note data elements and state whether they would be expected to conform with Benford's law.
2. Sort the data by surveyor.
3. For each surveyor and each data element expected to conform with Benford's law, prepare a population distribution and a chart, using the first digit.
4. For each surveyor and each applicable survey element, compute the D-Statistic using the Kolmogorov-Smirnov metric.
5. Review each chart and data table prepared to determine if the data appears to conform with Benford's Law.
6. For data which does not appear to conform, how does the D-Statistic compare with the others?
7. Write a report explaining your observations, and provide conclusions and recommendations.

Example Data (Excel and Flat File)

Excel WorkBook Sheets BenSurvey100 - BenSurvey104
Tab separated file ben.tab

Scripts to do the Data Extraction

Description of Analytical Procedures Performed

First sort the file provided in Surveyor Sequence
This can be done by opening the file in Excel, sorting and then saving as a tab separated file, or else use EZ-R Stats for Windows Tools | Sort and specify the input file as "ben.tab", the output file as "Bensort.tab" and the sort key as surveyor.

Using Ez_R Stats for Excel, choose the menu option Tools | Benford.
Specify input file "ben.tab", output file "Benfordresults.tab"
Select Benford Type "F1" (first digit)
Variable to be tested will be StreetAddr, NumPeople, and ChurchDistance
By variable name is Surveyor.

Screen Shots of Analysis

Screen Shots of Output

Discussion of Results

A visual review of the charts below indicates that the survey results obtained by the first three surveyors very closely approximate what would be expected. However, the chart of the results for the fourth surveyor, number 103, are strikingly different. Thus, it is desirable to further review and verify the results obtained by surveyor 103.

Alternative Approach - Batch Commands / Scripts

The analysis can also be done using a command line:

sort /ezs/temp/sort.ezs ben.tab bensort.tab Surveyor

then run the benford analysis (lines wrap around)

CtrlBrk.exe -e/ezs/temp/ben.ezs -iben.tab -o/ezs/temp/benresultspeople.tab -vNumpeople -bSurveyor -mMAX type=F1
CtrlBrk.exe -e/ezs/temp/ben.ezs -iben.tab -o/ezs/temp/benresultsstreetaddr.tab -vStreetAddr -bSurveyor -mMAX type=F1
CtrlBrk.exe -e/ezs/temp/ben.ezs -iben.tab -o/ezs/temp/benresultschurch.tab -vDistanceChurch -bSurveyor -mMAX type=F1

The analysis can also be performed using scripts:

options obs=max
libname benford 'c:/temp/ben';
proc sort data=ben;
by surveyor;
out = benford.sortben;

proc benford data=sortben type=F1;
var StreetAddress;
by Surveyor;
out=benford.addr;

proc benford data=sortben type=F1;
var Church;
by Surveyor;
out=benford.church; proc benford data=sortben type=F1;
var NumPeople;
by Surveyor;
out=benford.people;

Charts of Surveyors Results Compared with Benford's Law

Surveyor Id
100
101
102
103