Transformative evaluation and project methodology

Transformative evaluation

The study will use a transformative paradigm, which “ focuses primarily on viewpoints of marginalized groups and interrogating systematic power structures through mixed methods to further social justice and human rights” (Mertens & Wilson, 2012, p. 41). The methodological assumption of this approach is engaging both the researcher and participants in dialogic (vs. predatory) and cyclical (vs. linear) processes using appropriate qualitative approaches, such as narrative inquiry, and quantitative methods, such as surveys developed through grounded theory, and taking account of the historical context, existing power relations, and multiple perspectives (Chase, 2005; Cannella & Lincoln, 2007; Guba, 1990; Guba & Lincoln, 1989; Mertens & Wilson, 2012).

According to Mertens & Wilson (2012), transformative evaluation builds on the constructivist principles of considering values in the inquiry process (those of the researcher and the stakeholders involved) and stresses the importance of empowering the research participants while addressing existing systems of oppression and privilege; in fact, one of its primary principles is being culturally respectful throughout the research process.

“Cultural competence in evaluation can be broadly defined as a systematic, responsive inquiry that is actively cognizant, understanding and appreciative of the cultural context in which the evaluation takes place; that frames and articulates the epistemology of the evaluative endeavor; that employs culturally and contextually appropriate methodology; and that uses stakeholder-generated, interpretive means to arrive at the results and further use of the findings” (Sen-Gupta, Hopson & Thompson-Robinson, 2004, p. 13, as cited in Mertens & Wilson, 2012, p. 167)

Research question

The project seeks to answer four interrelated questions: Who were the past beneficiaries of the Universidad del Valle (UVG)’s vocational program under study and its potential beneficiaries? What were the contributions and limitations of this program promoting violence prevention in Guatemala? What is the program potential to serve vulnerable populations originally excluded from the program? How can this and other vocational training programs for youth at risk be improved?

Research sample

Those involved in the research will secure access to the appropriate institutional placements and settings to interview the targeted youth and minors. This will include nongovernmental organizations (NGOs) and public institutions serving youth at risk. Three at-risk youth groups will be included in this study:

  1. Youth from violence-prone areas. These will be past program beneficiaries. Some are expected to be young adults at the time of data collection
  2. Minors returned from the U.S. These can be unaccompanied minors who traveled, were detained and deported from the U.S. Some are expected to be young adults at the time of data collection.
  3. Children under the care of the state and residing in public or private orphanages. These will include children within the welfare system (in Casa Hogares) or institutionalized children in the facilities of non-governmental organizations. Some are expected to be young adults at the time of data collection.

Project methodology

The project proposes to involve a wide range of stakeholders, i.e., current and potential beneficiaries, and the agencies presently or possibly involved in the project. Specifically, semi-structured interviews and focus groups will be conducted with beneficiary students (adult youth, graduates from UVG’s Highland Campus), their families or guardians, the hosting families in Solola (with whom students stay), the teaching faculty, the UVG administrators, and the private and government institutions that have provided some type of support to the program beneficiaries.

The approach will include the following components:

(1) Stakeholder consultations will be carried out prior, during and after the research is conducted. These consultations of 30-60 minutes are informing the project design, implementation and the presentation of results. A total of 5-10 stakeholders will be consulted in the U.S. and in Guatemala.

(2) Face-to-face interviews will be conducted. A minimum of 5 youth for each of the 3 vulnerable groups plus their caretakers and providers will be interviewed. An estimated total of 45 interviews will be conducted. Each interview will last approximately 30-40 minutes.

(3) Three focus groups of 60 minutes will be carried, one per vulnerable group. These will be carried out when the interviews have been completed to discuss some of the preliminary results.

(4) During February to April 2016, a series of three seminars will be carried out. An estimated of 20-25 participants (including speakers) will participate in each of these events, which will last 2 hours each.

(5) The research will involve two bilingual UVG students of the PhD program in Applied Psychology as research assistants, who will be compensated by the UVG’s College of Social Sciences (CSS). Prior and during data collection, the PI will train the Guatemalan PhD students selected by CSS in the use of the transformative evaluation and in associated research methods, including how to carry out interviews and develop focus groups. The UVG PhD students assisting this research will carry out semi-structured interviews and focus groups with the two potential target populations of this program assessed: unaccompanied minors returned from the U.S., and minors or young adults currently or graduates from the child welfare system.

Other methodological elements

Data will be analyzed in Spanish in conjunction with the UVG students involved in this research project. Research results, conclusions and recommendations will be written in Spanish. Project results will be translated into English and disseminated in both languages to reach a broad audience of potential funders, journals, and policy makers in Guatemala and the U.S.

During summer and fall 2015, the Fulbright scholar Dr. Carmen Monico engaged in consultations with U.S. policy advocates and providers of services to youth at risk, particularly of those servicing or concerned with those three non-exclusive groups in Guatemala, i.e., youth from violence-prone regions, minors returned from the U.S. or children under protective care due to abandonment and abuse. These conversations have informed the project design.

In spring 2016, the project will facilitate the development of a series of discussion seminars with academics (faculty and students) and violence prevention officers and practitioners in Guatemala. The seminars will be carried out between February 2016 and May 2016 at the UVG. The three seminars will cover the following topics: (1) youth at risk and violence prevention programs in Latin America; (2) transformative evaluation of a vocational training program of the UVG benefiting youth at risk; and (3) feasibility of educational programs as an alternative for youth at risk in Guatemala. The proceedings of these seminars will be recorded and integrated in data analysis, data interpretation, and future data reporting.


American Council Immigration Impact (2015). One Year Later: Government Officials Request Lawyers for Immigrant Children and More Judges. Retrieved from

Cannella, G. S., & Lincoln, Y. S. (2007). Predatory vs. dialogic ethics: Construction an illusion or ethical practice as core of research methods. Qualitative Inquiry, 13, 315-335. doi: 10.1177/1077800406297648

Catholic Legal Immigration Network (undated). The Impact of Unaccompanied Children on Local Communities – Frequently Asked Questions. Retrieved from

Chase, S. E. (2005). Narrative inquiry: Multiple lenses, approaches, voices. In N. K. Denzin & Y. S. Lincoln (Eds.), Sage handbook of qualitative research (3rd ed., pp. 651-679). Thousand Oaks, CA: Sage.

Gordon, I. Mother Jones, July/August 2014 Issue. 70,000 Kids Will Show Up Alone at Our Border This Year. What Happens to Them? Officials have been stunned by a “surge” of unaccompanied children crossing into the United States. Retrieved from:

Guba, E. G. (1990). The alternative paradigm dialogic. In E.G. Guba (Ed.), The paradigm dialogic (pp. 17-27). Newbury Park, CA: Sage.

Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.

King, K. (2009). Scholarship of Teaching and Learning and the IRB. Retrieved from

Mertens, D. M. & Wilson, A. T. (2012). Program evaluation theory and practice: A comprehensive guide. New York, NY: The Guildford Press.

The White House (2014, July 25). Joint Statement by the Presidents of the United States, Guatemala, Honduras, and El Salvador. Retrieved from:

United States Customs and Border Protection (2014, 21 Nov.). Southwest border unaccompanied alien children. Retrieved from