“It’s biased”: the defense by the head of the CMIE of the CPHS investigation elicits a new critical response from Jean Drèze, Anmol Somanchi
In fact, Vyas’ reply confirms that the survey is likely to be biased, if only because it starts from the “main street” of each sampled village. Even if the sample size often forces the survey to go beyond the main street, as Vyas points out, the fact remains that small street households will be under-represented. This may or may not be the main source of bias, but it is certainly a smoking gun.
We plead guilty to rounding the CWSF estimate of adult literacy (aged 15-49) in urban areas from 99.6% to 100%. But even 99.6% stretch the imagination, as do many other numbers we’ve presented to illustrate the bias. The attached graph, comparing CPHS illiteracy rates among adult women (for rural and urban areas combined) with corresponding estimates from the Fifth National Family Health Survey, NFHS-5, speaks to itself. -even. To put things in a global perspective, the adult literacy figures from the ESCP would imply that India has suddenly become a world leader in this field among low- and middle-income countries, overtaking China, Vietnam. , Sri Lanka and other countries with record investment in mass education.
As we have pointed out, there is also a major divergence between CPHS and NFHS-5 in terms of the ownership of household assets. According to recent tweets and personal communications from Sanjay Kumar, director of the Center for the Study of Developing Societies (CSDS), the recent CSDS-Lokniti election surveys are also at odds with the CPHS data in this regard, and much easier to read. reconcile with the NFHS- 5 data. By way of illustration, the proportion of households without television in 2019, in the 11 main states where NFHS-5 is on track, was 29% according to NFHS-5 and 25% according to the 2019 national election survey. of CSDS-Lokniti, but only 6% according to the CPHS. Similar contrasts apply to refrigerators, toilets, etc. In short, there is a worrying trend of underestimating poverty in the ESCP, compared to other national surveys.
The representativeness of the CWSF data cannot be deduced from the survey methodology. It should be scrutinized by comparisons with other credible sources. We have drawn attention to important discrepancies with the National Family Health Surveys, Periodic Labor Force Surveys, the Census and now the CSDS-Lokniti Surveys. Vyas maintains that the CPHS is more reliable than other sources, but it’s unclear why this would be the case.
We have pointed out that the under-representation of poor households in the CWSF could increase over time. This is an important question, as one of the main purposes of the CWSF is to assess time trends. The survey presents an optimistic picture of economic progress in recent years, with, for example, the share of households earning less than Rs 1,000,000 per year in the CWSF sample dropping from 31% in 2014 to 6 , 6% at the end of 2019 while Vyas observes. It is possible, however, that this rapid progress is at least in part driven by the growing under-representation of poor households in the ESCP data. Even if the bias does not increase over time, it is likely to affect these observed trends. This is only one illustration of the importance of the representativeness of the sample. Of course, this does not preclude making good use of CWSF data for certain purposes.
In short, we do not see the need to take anything away from what we have written in light of Vyas’ reply. This is not to dispute the value of the CWSF as a regular large-scale national survey or to ignore the difficulties of carrying out such a survey. We ourselves have learned a lot from the CPHS. If the CMIE is committed to further improving the survey method, as Vyas assures us, nothing of the sort. Hopefully, researchers and other users of the CWSF data will also help address these issues and, in the meantime, exercise appropriate caution.