The Impact of Vocational Training for Young Women in Delhi

One popular solution to unemployment is to provide the unemployed with more skills through training. However, the impacts of vocational training in developed countries have been at most modest. There is much less well-identified work looking at impacts in developing countries, exceptions being randomized evaluations of programs in the Dominican Republic and Colombia. A new paper by Pushkar Maitra and Subha Mani adds to this evidence by reporting on an evaluation of a vocational training program for young women in one city in urban India.

Training details: the training was a 6 month course on stitching and tailoring services, provided by a collaboration between two NGOS – Pratham (well-known to impact evaluation readers for its work in education experiments in India) and a much smaller NGO called Social Awakening through Youth Action. Training was up to two hours per day, 5 days per week, and was offered in two low-income neighborhoods of New Delhi.

Participants: Participants had to be a woman aged 18 to 39, have at least 5 years education, and apply for the program. 658 Individuals applied, and two-thirds were randomized to the treatment group, and one-third to the control. At baseline the average participant was 22 years old, 51 percent were from Scheduled Castes, and only 5 percent had any form of employment. Participants had to pay 50 Rs per month to attend, and if they attended all 6 months of the program, got back 350 Rs. Fifty-five percent of those selected for treatment completed the program to get a certificate.

Follow-up survey: A single follow-up survey was conducted six months after the completion of the training, and was able to interview 504 of the 658 participants (76.6%). Attrition was slightly higher (26.4%) for control than for treatment (22.0%), although the attritors who answered the baseline survey (10% were not interviewed at baseline) do not look different on observables from the non-attritors. The authors inform me they have just finished a second follow-up 18 months post-program.

In addition, the authors try and measure a number of preference and behavioral characteristics such as risk aversion, competitiveness, and discount rates through lab games. Consistent with several other studies (e.g. Berge et al’s business training work in Tanzania), they have trouble getting this done for their full sample and only end up with data on this for 135 individuals, limiting how useful I view the part of the paper using this information.

So what do they find? Remembering these are 6 month post-training impacts, they find:

·The ITT impacts ignoring attrition show increases in employment, which are large in percentage terms relative to the very low base rate in the control group, but relatively modest in absolute terms: Any employment doubles from 6% in the control group to 12% in the treatment group; self-employment goes up 5.1 percentage points from the 1.2% rate in the control group.

The total cost of the training to the NGO was 1810 Rs per person, so if these increases in earnings continue over time, the total cost of the program can be recouped in less than two years.

Issues and lessons

·How you measure employment matters: one would think that labor market evaluations should have an easier job knowing what their end outcome should be (employment) than programs with multiple end goals (e.g. CDD programs). However, as I have been finding in several evaluations I am working on, even “employment” is not necessarily well-defined. The authors look at 5 different measures (casual wage employment, full-time employment, self-employment, any employment, hours worked). They find no significant impact on full-time employment, but significant impacts on the other measures.

·How long do we need to see impacts? One big question is whether the impacts persist or not. In a context like this, where so few women work, and those that do work less than full-time, attachment to the labor market is not that strong. So these jobs created might be temporary. Alternatively, they may have just sped up how quickly people find jobs, so that the control group could catch up over time. An alternative leading to bias in the other direction may come from comparing the control group, who have had 12 months to find work, to the treatment group, who may have only 6 months to find work if they don’t look for work while training. Measuring impacts over multiple horizons seems important for these types of studies. The new 18 month follow-up data will thus be very interesting to see.

·Is attrition driving the results? The attrition rates of this study are not that much higher than those in the Dominican Republic and Colombian studies. But rates of 15-24% attrition are a big concern when the program effects one is looking at are only in the order of 5-6 percentage points – clearly any Manski-style bounding approach would lead to very wide and uninformative confidence intervals. Lee bounds or the bounding approach discussed several weeks ago here may help narrow these intervals while still accounting to some degree for attrition.

·Context, when is skills the constraint, and displacement: The authors note the context is one where India’s economy has been growing at 7 percent or so, meaning that employment opportunities are opening up but many youth lack the right skills to access them. But it is unclear to me that stitching and tailoring are really the skills where lots of jobs are likely to be opening up in a growing economy, and so a concern is that any employment found is just the result of displacement – those who got the training getting jobs that would have otherwise gone to other people.

Given the importance of employment as a policy issue worldwide, and the lack of impact evaluations for active labor market policies in many regions, studies like this are very welcome, and hopefully further work in this area can build on some of the lessons from this one. One point raised in correspondence with the authors is the difficulty in trying to obtain measures on attitudes and preferences like confidence etc – bringing people to a lab can be expensive and have high attrition, but while individual games to measure risk aversion and some other preferences can be done in the field, I don’t know of successful ways people have done games that involve others without having to bring people to a lab – anyone have experiences to share?

Comments

David,
many thanks to you for sharing these interesting evaluation results, this is really very useful as evidence from low income countries is scarce.
A few thoughts on the issues you pointed out:
- Possible temporary nature of the employment: I wonder if even a temporary work activity by the women/mothers would yield positive effects on their self-esteem, education of their daugthers/children, work attitude/expectations of daugthers, and intra-family dynamics. As you mention, it would have been so interesting to get to know more about behaviors and attitudes. Too bad this did not work out, but I am sure that the effects of this kind of employment go way beyond the pure income/employment effects.
- Displacement effect: You rightly point out that we don't know much about this effect. However, my sense is that in a growing economy even traditional trades and corresponding services would expand. Also, India has a large textile sector, so couldn't we assume that because of the skilled labor working in these trades, the sector would expand? How can we ever know?
Thanks again for sharing and interpreting these results, keep it coming, please!

David:
The real takeaway for me was the number they reported for job search (Table 6) 0.066, in other words six percentage points more likely to report that they were looking for a job. While this reported as an outcome, one should treat (pun intended) this more as an intermediate outcome leading to greater employment/earnings etc, and one that training programs need to be focus on. How exactly did the training improve their job search skills? By instilling confidence and stronger sense of self esteem (they suggest this and correlate self reported higher confidence) which potentially would imply searching both more extensively and intensively given higher reservation wages? Or did their search become more effective given the expansion of their social network skills or link them up with potential employers given the informal/formal contacts of the training organizations themselves?
Visit any garment factory, particularly those catering to export sector, in the industrial parts of New Delhi, and you'll see a preponderance of men at all levels, especially at the shop floor. Add the stories of women workers being harassed at work and on the way and back to work and you have a pretty daunting challenge for these women in getting paid work. So what the authors should revisit is how the training and the support networks created through the NGOs helped in negotiating positions for these women. How were potential employers, especially those willing to offer some form of contractual employment approached in the first place? Did those women managed to secure such type of employment or paid apprenticeship find it easy to adjust to this change? For others how did they find clients ? As such the authors would be recommended in their follow-up to go beyond surveys by incorporating a more systematic use of qualitative methods to explain these findings particularly in understanding how the observed training effects arose in the first place (skill accumulation versus matching). In this instance this might include one-on-one or group interviews with purposive sampled women who attended and those didn't, and eliciting answers relating to asking their experiences in finding, and maintaining any form of employment, the nature of the support received from parents, interactions with employers or clients.