Colombia Ex-Combatant Project: Case Study
Using Python tools to improve policy research
Overview:
Dr. Olver Kaplan, associate professor of International Studies at
University of Denver's Korbel School, is on a mission to illuminate
the root causes of human rights issues around the globe. He worries
that efforts to reintegrate ex-combatants,
those who have been part of armed rebellion against the
government, are falling short. Access to legal employment is a key factor in
determining whether these individuals will effectively reintegrate
into society or turn back to violent forms of protest. In an effort
to improve the
reintegration process in Colombia, Dr. Kaplan organized a study on hiring bias against
ex-combatants. Though quantifying bias can prove to be difficult,
Dr. Kaplan believes we can shed tremendous light on the extent of
this issue, given that we can find the right data.
I joined the study as a research associate in January 2022, brought
on by a professor at UT for my experience with conducting field
research. By the time I began working with the DU team, they had
already discovered two potential sources of data for the experiment,
both of them job databases for those seeking employment hosted by
Colombia's Agencia Pública de Empleo — think of these like a
government-funded LinkedIn and Indeed. After obtaining appropriate
login information from our Ex-combatant participants, they began the
process of building a dataset for one of these websites,
SENA.
However, trouble soon arose.
Webscraping
a github link will be here
Our original web scraper became obsolete about a month into the
project's pre-launch phase when we discovered internal
restrictions on which jobs a particular candidate could apply to.
Because our dataset turned on the outcome of a
particular job application — either accepted or denied — we needed
to be able to actually apply to the jobs we'd scrape from the
site. Thus, I determined that a second scraper needed to be built
to scrape our second site: Buscador.
Pictured here is the scraper in action. Using the appropriate login information, the scraper logs into the Buscador site and scrapes a list of jobs that the candidate pre-qualifies for based on their profile conditions. These conditions include former employment and education, location, work preferences, and others.
Webscraper sped up for concision