Training without a Soul

By Claudio de Moura Castro, Positivo, Belo Horizonte, Brazil.

Brazil shared the traditional disdain of Iberian countries towards working with one`s hands.

senaiSENAI had a decisive role in reversing prejudice and creating a well-prepared labor force. Being a private agency, run by industrialists and funded by a levy on industrial payroll, it had what it takes to face this arduous task of going against the grain of local culture. German and Swiss role models were grafted into the DNA of the institution. It was training with a Swiss soul, since this was the nationality of the first director in charge.

The system has been around since 1942 and remains effective, as well as impressive. However, there are signs that the sailing is not as smooth as one could think; there are conflicting signals.

Candidates for the “classic” occupations, such as machinists, carpenters, brick masons and several others are shrinking. Dropouts are more numerous. And the proportion choosing to work in the occupations learned is lower.

At the same time, there has been a massive increase in the pay of these same occupations, far more than for any others. It is somewhat bizarre, more pay and less interest in these same occupations.

What is going on? Funded by SENAI, I am beginning an inquiry into this puzzling situation.

One possible reason is the combination of more vacancies for higher education and higher levels of schooling of the young work force.  Many more students reach or finish secondary education, making them closer to the enticement of a magic university diploma. In addition, there has been an expansion in fellowships and student loans, making the temptations more reachable. Even if youth fail to enter higher education, its magnetism deflates the possible interest they could have for trade training.

Another hypothesis is the growth of the service sector. Regardless of pay, work is lighter and has greater prestige. As someone said to me: it is better to be a security guard in a shopping center than a machinist.

In addition, there is a third persuasive cause and it is inside SENAI.

In the last few decades, SENAI considerably expanded its secondary technical courses. It also created two-year post-secondary programs. In parallel, it is boosting its R&D and innovation activities. The result is a change in the center of gravity of the institution. The new activities require a lot more masters and PhDs and hardly any craftsmen. More money and discussions focus on these higher-end activities. Nothing is wrong with this new emphasis. It is a mandatory direction to be taken, considering the technologies deployed by modern industry. But the unavoidable consequence is a decreased attention to trade training.

Turners and woodworkers are taught as before. But, perhaps, something is missing. Are the trade instructors as proud as they were? Has their leadership towards the students slackened? Is there less effort put into training them? Is their heart and soul still in it?

I visited a very fine school of natural gas technology, with a serious applied research agenda. After the visit I realized that the highly sophisticated managers did not show me the workshops where students were trained to convert gasoline automobiles into gas combustion. One cannot imagine that students and instructors fail to perceive that their position is seen as inferior, considering that their workshops did not deserve being shown to visitors.

When I first compared SENAI with its European counterparts, it struck me that it was using its facilities and instructors to offer courses of widely different levels to a broad clientele of students. It could be training in a simple trade or a post-graduate program. That seemed a very efficient way of deploying resources. In comparison, European institutions catered to much narrower sets of students.

In hindsight, we can see that mixing high and low status occupations in the same school can be detrimental to those at the bottom end of the spectrum.

This is not a hopeless situation. By the same token that SENAI was able to prepare a generation of proud workers, when conditions were much more difficult, it must be relatively easy to pay a lot more attention to convert the soul of the students, not just deliver the curricula and operate workshops.

Be that as it may, these are conjectures, based on observation, rather than systematic research. The forthcoming study will confirm or deny the above speculations.


Claudio de Moura Castro, Positivo, Belo Horizonte, Brazil. Email:


Posted in Uncategorized | Tagged , , | Leave a comment

Back to the ‘Vocational Education’ Drawing Board: Do we Need Some Serious Re-visioning?

By Salim Akoojee, South Africa and Hong Kong.


Vocational education 110 years ago (Image from Kansas State Manual Training Normal School, 1904)

An August 2014 article in the Economist referred to the vocational sector as being the ‘detritus of an industrial era rather than the handmaiden of a new economy’. Citing the ‘twin curses of low status and limited innovation’, the Economist cites the McKinsey report  in which students in four out of seven European countries surveyed were discouraged from vocational education as a result of the ‘disorganisation’ and ‘lack of prestige’ of the sector compared to universities. If we were honest, we have to admit that this is indeed the case in a number of emerging economies as well. Indeed, poor vocational education enrolment is compounded by even more serious disengagement from the world of work. While we could contest the wisdom of blaming the sector for this lack of engagement with the wider economy and its disorganisation and decay, there is perhaps a need to identify where this comes from and perhaps more importantly, why and how it can change so that the sector can take its rightful place as a legitimate sub-sector of the national education and training systems.

It is clear that the various skills challenges that a country faces does not rest entirely with the vocational sector itself. The fact that it has not kept pace with the changing economic realities (as initially envisaged) suggest that some serious re-thinking is necessary. This is especially so in emergent economies with the deep seated social and economic challenges that have potential to create considerable upheaval. We therefore have to get back to the drawing board regarding the key question of purpose, mission and vision of vocational education, referred to in the international discourse as Technical, Vocational Education and Training, or more recently Technical and Vocational Skills Development (TVSD).

Let us begin with the purpose identified in the international discourse during the colonial era, underpinned as it was by a link with an economy crafted on the basis that the benefits of industrialisation were expected to flow to the colonies and that the skills systems needed to be responsive to this new reality. The current discourse has, of course changed. The sector is considered to be tasked with the responsibility of skills for work and further learning so that it advances the ‘human, social and environmental aspects of development’ (see UNESCO, 2012). However, the reality is that it is still embedded in a formal industrial economy discourse, and stuck in a time-warp that has not been radically unseated. It therefore carries its label of the handmaiden of an industrialised economy as a crown that has lost its glimmer.

This means that the traditional responsiveness of the sector to develop a mid-level skills corps for a widening industrial labour force, primarily driven by mining/agriculture extraction and manufacturing needs to be radically redefined as economies have transformed. The fact is that the very nature of the economy and labour market has changed.

The challenge of a shrinking formal technical employment market means that a new story needs to be written regarding notions of development.[1] Jobs are a central concern for those in developing countries, where obtaining decent work and employment is slowly becoming more and more elusive. Since the industrial model and the expansion of the formal economy is less likely to be the panacea promised by some economists, the reality has to be understood and taken on board.

It is no wonder, therefore, that the TVET sector, as traditionally crafted, has ceased to be the handmaiden of an economy, when the very notion of economy has changed over time. In emerging economies, the lack of a vibrant and expansive formal economy means that if the sector is still to be responsive to national development prerogatives, it has to take on board the new reality and change accordingly.  As a recent report on African TVET pointed out, ‘…at least three out of four jobs are created by the self-employed and micro/small business entrepreneurs, who are the principal drivers of the local, national, and sub-regional economy’.[2] The reality of this “new” development and its absence in global discourse suggests that a new story has to be written and taken on board in the emerging world.

In both emerging and high income contexts, the danger of not crafting a responsive TVET agenda for a new era has the capacity of undermining its core purpose and leads to disconnect.  Current purposes must respond to the objective conditions of inclusive economic and social development that many of those in the emerging economies face. This means that the sector has to respond more effectively to the national development discourse; whether it is infrastructural development, developing small business; entrepreneurship development or enabling targeted national resources. Ensuring that the sector supports the advancement of achieving ‘sustainable livelihoods’ for all in support of the drive for inclusive development has to be a key feature of its overall thrust.

Closing thoughts

Understanding that the purposes to which TVET has subscribed over the past few decades and how that reality has changed is an important way of beginning the process of renewal. TVET needs to respond to the economic and social development trajectory in individual countries. Perhaps the notion of TVSD, which incorporates all of the varied forms of skills development and argues for widening the basis for skills development outside of the formal (traditional) structure, is an important start to this renewal. There is clearly a need for re-thinking the purposes of the mid-level skills development sector as it responds to various sectors not traditionally responded to.

Yes, we could be deceived into thinking that vocational education is ‘detritus’ of that bygone industrial era and if understood in that way, there will be a strong case for consigning it to the rubbish heap of history. But this misses the point about its new mission, vision and purpose. As a sector that engages the various skills and challenges that face societies, it can serve as our saviour for the beginning of a brand new era of inclusive development. TVSD should enable and enhance the meaningful participation of those that other education and training forms are unwilling, or perhaps unable, to respond.  As Thomas Piketty insightfully reminds us, ‘It is obvious that lack of adequate investment in training can exclude entire social groups from the benefits of economic growth’ (Piketty, 2014). Thus the ‘diffusion of knowledge and investment in training and skills’ needs to take centre stage if we are to resolve the key contradictions that we are all aware of. If we are to respond to the reality of a world where rampant inequality, poverty and unemployment is likely to be considerably exacerbated by the lack of skills, there is a need for ensuring that all our education and training entities are appropriately directed to respond to the  real issues of national development. Skills have to be the solution, rather than the basis for the societal inequality that we know needs to be resolved at every level.

Salim Akoojee is an Adjunct Associate Professor at the University of the Witwatersrand, Johannesburg. He currently lives in Hong Kong. Email:


Piketty, T. (2014). Capital in the Twenty-First Century. (T. b. Goldhammer, Ed.) Cambridge (Massachusetts) and London (Engliand): The Belknap Press of Harvard University Press.

Standing, G. (2011). The Precariat: The New Dangerous Class. London and New York: Bloomsbury Academic.

The World Bank. (2012). World Development Report, 2013: Jobs. Washington: The World Bank/ International Bank for Reconstruction and Development.

UNESCO. (2012). Current Issues and Trends in Technical and Vocational Education and Training: A UNESCO Report. Paris: UNESCO

[1] In his regard, it might be particularly useful to challenge existing notion of development that the World Development Report (2012) espouses when it points out that, ‘Economies grow as people get better at what they do, as they move from farms to firms, and as more productive jobs are created and less productive ones disappear’ (p.2). The notion of ‘less productive jobs’ needs perhaps to be redefined as  notions of innovation are cast in terms that advance more capital investment and technology and fewer jobs.

[2] The ICQN/TVSD 2014 Ministerial Conference, “Providing Africa’s Youth with Skills and Training for Jobs”, Abidjan, 21, 22 and 23 July 2014, Golf  Hotel / Riviéra, Summary Conclusions on the ICQN/TVSD Country Reports.

Posted in Uncategorized | Tagged , | Leave a comment

The Post-2015 EFA Agenda: UNESCO and the New Global Education Network

By Maren Elfert, University of British Columbia.

CaptureThe run-up to the next round of Education for All (EFA) and development goals lends itself to a reflection about the political-economic underpinnings of the discussion about the future of EFA. The report of the high-level panel of eminent persons on the post-2015 development agenda emphasizes the role of the private sector. The document points out that “the massive investments that will be needed for infrastructure in developing countries [will require] …new ways of using aid and other public funds to mobilise private capital” (p. 3). It further stresses the role of “private philanthropists” and the importance of business “as an essential partner that can drive economic growth”, because “large firms have the money and expertise to build the infrastructure that will allow all people to connect to the modern economy” (p. 11). Also “social impact investors” are mentioned, whose “efforts can be sustainable over time”, “because they make money” (p. 11).

The Global Education First initiative (GEFI) launched by the UN’s Secretary-General Ban Ki-moon in 2012 as a five-year campaign “for the final push” towards the achievement of the second Millennium Development Goal, universal primary education, strongly builds on partnership with business and makes an effort to get the corporate sector on board by using a discourse it can understand, such as “education develops human capital required to create a skilled workforce, improve productivity and drive business growth” (p. 8).

UNESCO’s post-2015 position paper

In sharp contrast to the above discourse, the position paper on the post-2015 agenda put forward by UNESCO’s Executive Board at its 194th session touts the state as the “custodian of education as a public good” (p. 2). It is worth quoting the entire paragraph, which appears under “guiding principle”, sub-item “fundamental principles”:

Education is a public good. The state is the custodian of education as a public good. At the same time, the role of civil society, communities, parents and other stakeholders is crucial in the provision of quality education.

The word “private” does not appear in the list of the “other stakeholders”, neither does it appear elsewhere in the document. After review by the EFA Steering Committee and the Task Force for the post-2015 Education agenda, the above paragraph remains untouched in the revised document (p. 3), but the term “private” appears in a footnote (p. 7) and in a list of “a broad coalition of partners for education beyond 2015” towards the end of the document (p. 10). Both – the total absence of the term “private” in the Executive Board draft, and the rather marginal mention of the private sector in the revised document – are remarkable given that the role of the private sector in education is gaining ground in the debates on the post-2015 EFA agenda.

How is this position paper compatible with UNESCO’s alliances with the private sector, such as its partnerships with Nokia and the Business Backs Education campaign, which was launched in March 2014 at the Global Education and Skills Forum held in Dubai? This event assembled a strange mix of philanthropic foundations, businesses, international organizations and high-level former politicians and proponents of the “Third Way” such as Tony Blair and Bill Clinton. One item on the agenda in Dubai were “social innovation” projects such as social impact bonds, which are about to replace the traditional charity model. They allow governments to retreat from their social responsibilities, save money and leave it to businesses to set the agendas of what is considered “social innovation”.

The basic concept of a social impact bond is that a service provider, usually an NGO, will raise money from private investors in order to fund a social program. The private investors will be reimbursed by government if they can prove that the program has been effective. The problem with this model is obvious: it puts even more emphasis than we already have on measurable results, at the detriment of long-term sustainable programs.

What does UNESCO stand for?

This contradiction between UNESCO’s partnerships with the private sector and the post-EFA position paper points to larger problems. First, it is increasingly difficult to tell what UNESCO stands for. On the one hand, there are staff members such as those who drafted UNESCO’s post-2015 education position paper who are committed to UNESCO’s humanistic approach to education which is laid down in UNESCO’s constitution. On the other hand, UNESCO pursues “strategic partnerships”  with private corporations in order to secure its survival at a time when it has to keep house with a budget reduced by the lack of United States’ membership dues. Another reason for UNESCO’s engagement with the private sector is the increasing difficulty to find aid money for basic education, which is one of UNESCO’s priority program areas. In a recent speech, UNESCO’s Director-General Irina Bokova deplored an annual $26 billion underfunding of basic education (p. 5).

Although UNESCO’s interest in these new partnerships is understandable given the situation of the organization, these alliances are not as unproblematic as they are often presented. An organization such as UNESCO and a private corporation represent two very different paradigms. While the goal of a business is to maximize profits and therefore to appropriate material wealth to the exclusion of others, UNESCO’s mandate in education is based on principles of equality, sharing and participation. While for businesses education is a commodity, for UNESCO it is a human right. There is not much that these disparate ontologies have to say to each other.[1]

Who shapes the global education agenda today?

Second, and more importantly − who shapes the global education agenda today? This is increasingly difficult to tell given the surge of “unauthorised actors” (Swyngedouw, 2014, p. 127, referring to Ulrich Beck) in the field. UNESCO’s constitutional partners are governments and NGOs. Relationships with the private sector are briefly mentioned in UNESCO’s constitution and are only foreseen if those institutions “pursue goals that are in conformity with UNESCO’s ideals” and “are entirely non-profit-making”. UNESCO aligns itself with new networks or “heterarchies” (Ball, 2012), including businesses, but doesn’t seem to have internalized this change in position.

UNESCO changes because its traditional partners change. Governments are reconfigured as one source of authority among many in a new global education network. UNESCO, with its rights-based approach, serves as “conscience” and gives legitimization to corporate actors. In turn, UNESCO’s identity will be affected, and the organization needs to get prepared for this. Ultimately, the new private actors in “global governance” may undermine even further the role of international organizations such as UNESCO.

Other References

Ball, S. (2012).  Global Education Inc. New policy networks and the neo-liberal imaginary. Abingdon, UK: Routledge.

Swyngedouw, E. (2014). Where is the political? Insurgent mobilisations and the incipient “return of the political”. Space and Polity. 18(2), 122-136. An earlier version of this paper is available here.

Maren Elfert is a PhD Candidate in the Department of Educational Studies at the University of British Columbia. Her research interests include UNESCO’s education work, in particular the concept of education as a human right. E-mail:

> See also Maren Elfert’s earlier NORRAG Blog post on ‘The Post-2015 EFA Agenda: The Role of UNESCO’

[1] This paper has benefitted from a conversation I had with Alexandra Draxler.

Posted in Uncategorized | Tagged , , , , | Leave a comment

Beyond Armed Conflict and Emergency: the Role of Education and Training in Tackling Urban Violence

By Jovana Carapic, and Luisa Phebo, Conflict, Violence, Education and Training (CVET) Programme, NORRAG.

Urban spaces are going to be the locus of future armed conflict and organized violence. The signs are inescapable.

One reason for this is that the nature of armed conflict is changing. Traditionally conceptualized as conflict between or within states, the number and intensity (in terms of battle deaths) of armed conflict has decreased since 1990. Although armed conflict is not going to disappear – as illustrated by the recent events in Iraq, Libya, Syria, and Ukraine – evidence suggests that it is no longer necessarily the most important source of insecurity affecting the majority of individuals around the globe. Instead, ‘everyday’ forms of lethal (often armed) violence in non-conflict settings accounts for the highest proportion of insecurity.

There has also been a change in where organized violence occurs. Although historically armed conflicts tend to take place in rural areas, contemporary warfare, insurgencies, and other forms of organized violence are increasingly concentrated in cities. The ‘urban’ nature of organized violence is also evident in non-conflict settings.  Indeed, when national homicide rates are disaggregated, it becomes evident that lethal violence is now more prominent in urban areas than rural areas, and that it disproportionately affects certain sub-sections of the population like women and youth.

Another reason is simply that the world’s population is rapidly urbanizing, especially in the Global South. In 2025, the urban population in Africa is expected to be larger than in Europe and Latin America; although half of the world’s urban population already lives in Asia, urbanisation in the region is expected to continue and lead to the creation of at least five mega-cities; finally, and even though Latin America is considered to be the most urbanised region in the world, the region is seeing a rise in city-to-city migration resulting in the proliferation of medium-sized cities.

Uneven urban development is yet another reason. Urbanization is often considered to be an indicator of development. This of course begs the question: whose development? Historically urbanization was closely associated with the formation of the state, consolidation of the use of force, and economic development. This was especially the case in Europe. Today, many low and middle-income countries which are undergoing the highest rates of urbanisation are not experiencing these benefits.

Arguably, one of the main reasons for this is that many developing countries are unable to deal with the rapid demographic shifts resulting from rural-urban migration. National and municipal government have only a limited amount of resources for fostering civic engagement, economic growth, and social inclusion in urban settings. To make it more explicit, while urban spaces are expanding (both in population and territory), this has not necessarily been met with an expansion in municipal institutions and capacities – especially those related to the provision of services such as security and education.

Consequently, political, economic and social policies tend to be unevenly implemented resulting in the ‘fragmentation’ of urban space which leads to cities becoming sites of socio-economic inequalities and characterised by pockets of insecurity. Together these changes have been found to lead to social and political crises – including in Brazil, Egypt, Tunisia, and Venezuela – but also have the possibility of transforming into full-blown armed conflict – like in Libya and Ukraine.

In other words, as a result of the various dynamics of global urbanisation – from where armed conflict is fought, to increased rural-urban migration, to uneven and fragmented cities – it is likely that in the future more conflicts, violent or not, are going to occur in cities.

The recognition that urban violence is a considerable threat to the stability and development of states and the wellbeing of their citizens has led international researchers and policymakers to tackle urban violence. This is often been done through the support and implementation of various armed violence prevention and reduction programmes (AVPR) or citizen security programmes. Central to many of these programmes is the provision of various types of education and training to youth – especially young males who are the main perpetrators (but also victims) of urban violence.

Arguably, the ‘theory of change’ behind many of these initiatives is two-sided: on the one hand, it aims to provide ‘tangible skills’ which allow individuals to find valuable employment (getting them off the streets and out of the reach of gangs); on the other hand, it aims to change behavior by providing ‘life skills’ which would foster productive civil engagement and reduce the resort to violence.

Thus, education – conceptualized broadly to include formal, non-formal and informal schooling, training, and learning – is potentially a central feature of any comprehensive effort to reduce and mitigate violent conflict in society and its potential escalation into collective violence. A significant portion of the globally implemented AVPR and citizen security programmes can be found within a handful of countries.

Brazil and South Africa are particularly good examples. In the past two decades, these countries have not only been characterized by high levels of socio-economic development, but also acute urbanization, inequality and high levels of lethal (mostly urban) violence. Thus, rather than being something new or a possible scenario for the future, urban violence is a ‘constant’ in these societies. This is important because it means that the development of various programmes and initiatives for dealing with organized violence have not been developed from scratch – like they are in conflict or post-conflict settings – but have organically evolved as the rate or urbanization and urban violence increased.

Consequently, Brazil and South Africa have been described as ‘laboratories ’for the prevention of urban violence, highlighting both scenarios of success and failure.

Experiences in these contexts allow for an examination of existing AVPR and citizen security programmes that focus on education (broadly conceived), but also for insights into their successes and failures (given the availability of evaluation studies and data). Insights from these two countries are important for they not only pave the way for current (and future) cross-national cooperation and learning, but also can feed into the international policy dialogue on the role of education and training for dealing with urban violence.

In this context, NORRAG’s most recent Programme of Work – Conflict, Violence, Education and Training (CVET) – aims to critically engage in the debate and examine the role of education in situations of urban violence. In view of the need to further understand what education programmes and initiatives are being implemented to tackle (either directly or indirectly) violence in the urban context, the programme will engage in a ‘mapping’ exercise of projects in Brazil and South Africa. The initiative aims to foster the development of research on the relationship between conflict, violence, and education which will be able to feed into the development of effective educational policies in urban settings – both from an international and national perspective.

Jovana Carapic is Research Officer for NORRAG’s Conflict, Violence, Education and Training (CVET) Programme. She has extensive expertise in the field of conflict studies, in particular on urban violence and the relationship between various forms of armed groups and the state. Her research has taken her to East Timor, Nepal, and Serbia. Email:

Luisa Phebo is part of NORRAG’s Conflict, Violence, Education and Training (CVET) team. She has previously worked at Viva Rio’s project management department in Brazil, and recently spent one month in Haiti visiting the organisation’ programmes as an independent researcher. Email:

Posted in Uncategorized | Tagged , , , | 2 Comments

Lessons From Chengdu (2): The Political Economy of Learning Metrics

By Trey Menefee , Hong Kong Institute of Education

pic3I argued in my previous NORRAG NEWSBite post, Lessons From Chengdu: The Case For ‘Open-Source’ Learning Metric Methods, that easily accessible non-proprietary learning metric methodologies would be a valuable asset to the international education community. What I argue here is that metrics themselves are not the problem. We are familiar with the adage that we cannot manage what we cannot measure, but to this we should add that metrics make for terrible managers. However, metrics go beyond just saying something – they seem to tell us what to do. An antidote to this is employing what might be called the ‘Scott Audit’, which answers three questions James C. Scott (2012) has proposed as a framework for navigating the politics and anti-politics of metrics. This post will use this auditing technique on our Chengdu research and conclude by showing how the issues addressed link directly with problems in the technical and vocational skills development (TVSD) sector.

  1. What is the relationship between the proposed quantitative index and the construct – the thing in the world – it is supposed to measure?

There is a risk that the metrics are measuring against the wrong norms or that they carry more symbolic weight than they can or should carry. Many rightly scoff at league tables associated with PISA which aim to offer a specific number showing how one school system is better than another. From another perspective, we see the Brookings-UIS Africa Learning Barometer doing an excellent job using learning metrics to expose the shallowness of another: enrollment rates. Their work shows that “37 million African children will learn so little while in they are in school that they will not be much better off than those kids who never attend school.”

In the Chengdu study, one of the earliest issues encountered was that Chinese English tests scores often have a low correspondence with a student’s actual ability to communicate in English. These tests measure what might be called “virtual English” – a reorganization of communicative English in ways that made it more accommodating to rote memorization and assessment. This fosters a justified concern amongst educators that focusing on communicative language teaching, usually considered best practice for language acquisition, might hinder student test scores.

Further, the merging of value-for-money approaches and performance-based development metrics lends itself toward breeding inequality within the unequal. There were substantial inequalities both between the private schools for migrants in Chengdu and within the classes. The easiest way for an NGO, school, or teacher to get measurable impact and ‘bang for the buck’ would be to ignore the struggling students and concentrate resources on the higher performing schools and students. A shallow analysis of learning outcome improvements, for instance an increase in the number of students ‘passing’ the Zhongkao English section, runs a high risk of identifying the most inequitable interventions as having the highest impact.

  1. Is a political question being hidden or evaded under the guise of quantification?

pic4Knowledge-based barriers to social mobility might be thought of as coming in two forms: intrinsic and extrinsic. Illiteracy is an intrinsic barrier when someone can’t read the instructions, signposts, or maps that might lead to a better livelihood. By and large, white collar jobs exclude the illiterate because literacy is genuinely necessary to perform those jobs competently. On the other hand, my hometown denied many African-Americans the right to vote through the use of literacy tests until it was outlawed in 1965. It would be morally indefensible to approach this problem with the solution of increasing literacy rates, no matter how valuable literacy might otherwise be. This is amongst the most difficult but important issues when working with learning metrics: determining whether the metrics are genuinely of intrinsic usefulness or whether they are instead being used extrinsically as tools for social closure.


A primary mechanism of social closure with learning metrics is a very specific type of equality – an equality in expectations. In context to English, it begins with unrealistic and largely uncontested curricular assumptions that all students are performing at roughly the same level. These expectations move into the classroom and create an environment where every student studies the same material at the same time, regardless of their command of the subject. The idea of equality is contorted, such that many schools and parents believe that offering remedial classes would be inequitable.

In practice, this amounts to the academic equivalent of a death march. Students furthest away from this arbitrarily invented academic norm find themselves abandoned by their schools, teachers, and textbooks a little bit more every day. There is little sense of progress for many students: the curriculum appears to march three steps forward with every stride they take. English, in this view, is deployed in China as a tool of social closure more than a subject of intrinsic value. These Chengdu students were failing by design. There are lessons here for any post-2015 goal requiring minimum learning outcomes. Who, exactly, will pay the price for failing to meet these new standards?

  1. What are the possibilities for colonization or subversion of the index, such as misreporting, feedback effects, or the prejudicing of other substantive goals?

Theodore Porter (1996, p. 43) poignantly noted that, “quantitative technologies work best… if the world they aim to describe can be remade in their own image.” There is a constant and largely unacknowledged dialectic that prioritizing specific outputs transforms the inputs. Whenever new learning outcomes are deemed important for success we find teachers and administrators transforming curricula to meet these expectations. Scott (2012, p. 116) aptly calls these ‘colonizing metrics.’

The importance of English in Chinese academic testing elevates it to compulsory, rather than an elective, subject. It joins a handful of academic skills that become ever more condensed as students matriculate. Questions mostly revolve around how to increase quality and improve scores, rather than challenging the fundamentals of the situation. It is through this mechanism that students, parents, teachers, and civil society get trapped in playing perpetual catch-up in a game that is designed to exclude three out four Chinese youth from higher education access.

While parents and students should be expected to adapt to these situations, researchers and institutions involved in collecting and analyzing learning metrics are in a position to identify and critique situations of education-based opportunity hoarding. At this, we too often fail. The conceptual purpose of PISA, TIMMS-PIRLS, or SAQMEQ scores rarely seem to rise above showing how much a country, district, or a school deviates from others. There is a danger that the post-2015 learning targets, should they come to pass, might begin a process of transforming schools systems around the world into machines that produce higher scores of questionable value that attract more aid dollars, higher league table rankings, and electoral advantage. The metrics themselves too readily sideline contentious politics and become managers.

Collecting internationally comparable learning metrics are a valuable tool for understanding educational inequalities if it can put politics front and center. They are dangerous when uncritically mixed with positivism and teleological assumptions about performativity. It would have been problematic if the primary rationale for collecting our Chengdu data was to only answer how scores might be improved, rather than to look at how learning outcomes were being used against students. The ‘Scott Audit’ used here is one way of ensuring that the politics of educational performance and development stay in focus.

This analytical lens highlights the proverbial elephant in the room: while there are two streams of secondary education in China, only one is seen as viable: the college preparatory track. Problems in China’s second stream, vocational  education, make it unattractive to even the most marginalized families. In this view, the problem was not low English scores but a lack of educational options for the 75% of students who won’t make it to university. What our learning metrics data provided was snapshot showing how many students were likely to fall off and the limitations of improving academic performance to improve equity. The better question to take from our data is not how to universalize high quality English education for a hundred million children, but “what’s an equitable Plan B?”

Trey Menefee is Lecturer at the Hong Kong Institute of Education and is currently working on the primary statistical report for the 19th Conference of Commonwealth Education Ministers, which focuses on educational quality and inequality. Email:

Posted in Uncategorized | Tagged , , , | Leave a comment

Lessons From Chengdu: The Case For ‘Open-Source’ Learning Metric Methods

By Trey Menefee, Hong Kong Institute of Education.


A few years ago I tried to learn a research technique called Agent-Based Modeling (ABM) that is used to understand complex adaptive systems. The learning curve for ABM was high because it required researchers to learn how to code. This learning curve was made a little less steep with, a website dedicated to “improving the way we develop, share, and utilize” these models. The ethos of the website was that people could share the codes for their models using a Creative Commons license, describe how it works and what the known issues were, and open their models for critique and improvement from a knowledgeable user community. Arizona State University and the Sante Fe Institute managed the website, ensuring a degree of quality control.

During my involvement with the Learning Metrics Task Force (LMTF) I came to see that the open-ended consultative style that Brookings and UNESCO UIS employed created something of a Rorschach Test. Nearly everyone exposed to the LMTF’s work saw something different in the inkblot. Among other things, what I saw was the possibility of something like for learning metric methodologies. It would be a place where discussions continued and methods evolve rather than having the settled finality of PISA. It would be free and have effectively no barriers to entry. Users would be people who might want to deploy these methodologies, and might need to make trade-offs between cost and rigor, and might need help in seeing the strengths and weaknesses of different assessment methodologies. Both results and methods would be shared by users.

Rebecca Winthrop of Brookings recently wrote that assessments should be seen as a “public good”, that “measures for the indicators recommended for global tracking must be considered a public good, with tools, documentation and data made freely available.” By definition, a proprietary good for sale on the market isn’t be a ‘public good’. The need for a non-proprietary and accessible toolkit like this was recently made apparent to me during a project working with migrant children in Chengdu. Condensing a very complex issue, migrant children in Chinese cities are denied the right to an education in urban public schools. Some can gain access, others enroll in low-cost private schools, and more than sixty million simply stay behind with relatives while their parents go off to work. Huizhi Social Work Service Center, an NGO that works with urban migrant populations, wanted to create a program to help some of these urban students. After consulting with stakeholders they identified English as the subject most in need of reform.

English is required for the middle school entrance exam (the Zhongkao), which was a make-or-break test to determine who would get to study for the National Entrance Exam (the Gaokao), which in turn determined higher education opportunities. My task to assist them with baseline observations and analysis was made difficult for two reasons: (a) comparative assessment methodologies for this demographic are in short supply, and (b) the contexts around student’s performance was political and needed data to be approached from this angle. The first issue is discussed in this post and the second will be addressed in a second post on NORRAG NEWSBite.

For the most part, English assessment is a business. While the Common European Reference Framework (CERF) provides an open and internationally comparable standard for assessing language proficiency, nearly all the tools for doing the assessment are either context-specific (e.g., for a school district) or proprietary (e.g., TOEFL, TOEIC, IELTS, etc.). The internationally comparable assessments are services for sale, not methods for researchers and organizations to use in contexts of poverty and inequality. That these tests have been developed for the market rather than the needs of educational planning or evaluation warped the assessments themselves. The market is almost exclusively focused on the upper end and occupied with issues like international university admissions. There are a dozen good options for assessing whether or not a Beijing adult speaks English well enough for an American graduate school, but there was little of relevance to the ten year old son of a Chengdu construction worker.

For me, this is the most compelling case for the LMTF: there was nowhere for Huizhi to find a high quality, internationally comparable assessment for their own staff to deploy. I was stuck in a maze of proprietary vendors. English assessment experts that I asked could think of no alternatives, usually designing their own assessments for specific purposes. At present, there are few good places to look towards to find the tools for the sort of ‘wake up’ Dr. Banerji called for on NORRAG NEWSBite in an August 2014 post.. TIMMS-PIRLS offers one open-ish model, though it’s arguably centralized, ‘one-size-fits-all’, and discourages innovation or adaptation to other learning subjects like English as a Foreign Language. The students already had such a purpose-built assessment in the Zhongkao, though its value in assessing actual English communicative competency is questionable (this is discussed in the next post on this blog). Ultimately I used a sample test for the TOEFL Junior, one of the few proprietary instruments developed for primary and secondary students, provided on the ETS website.

pic2We collected data in ninth grade classes in five schools in Chengdu. In my interpretation, the results showed that the vast majority of students were almost impossibly far behind in their English. On average, only one in twenty students crossed a threshold that could be considered lower-intermediate level (CEFR B1 equivalent, where they ‘can understand the main points of clear standard input on familiar matters regularly encountered in work, school, or leisure’). In some schools, almost half of the students told us that they found the assessment so difficult that they randomly guessed at answers without trying. These schools had systemically low quality with high turnover rates for teachers, crowded classrooms, and occasionally violent classroom management techniques. Using estimates of tertiary enrollment rates for this demographic, I estimated that out of five hundred students, Huizhi’s plan for increasing scores by 5% might send only six additional children out of 500 to university. The political contexts of what these results  mean are explored in the next post on NORRAG NEWSBite, Lessons From Chengdu: The Political Economy of Learning Metrics.

The ease, though ambiguous legal propriety of using ETS’s copyrighted material, of deploying the TOEFL Junior practice test allowed one of my students, Li Yixing, to conduct her own survey in Shanghai at a cost of only her time. Her preliminary findings suggested that similar students were doing significantly better than those I met in Chengdu, despite being two years younger. While I have doubts about the efficacy and ability to raise migrant English scores in Chengdu, these Shanghai metrics offer a new window into understanding what Shanghai is doing differently, how comparable the groupings really are, and ultimately whether there are lessons from either Shanghai or Chengdu that might be useful for groups like Huizhi. Were we to find a better assessment and share its methodology with other organizations and researchers, there is hope that we could create a more nuanced statistical snapshot of educational inequalities within China than anything that has yet been produced.

Trey Menefee is Lecturer at the Hong Kong Institute of Education and is currently working on the primary statistical report for the 19th Conference of Commonwealth Education Ministers, which focuses on educational quality and inequality. Email:

Posted in Uncategorized | Tagged , | 1 Comment

The Post-2015 Agenda: Examining the Youth Perspective and their Partnership Potential

By Nayantara Naik, Freelance International Education Consultant, New York, New York.

The voice of youth: Ahmad Alhendawi, the UN Secretary-General's Envoy on Youth

The voice of youth?: Ahmad Alhendawi, the UN Secretary-General’s Envoy on Youth

At sixteen, I attended a National Student Leadership Conference in Washington, D.C. Arriving at the program, I was skeptical. How could anyone teach leadership? Over the course of 10 days, I embarked on what was one of the most challenging journeys of my life. I began with grappling with basic principles of self-awareness and questions centered around: “Who was I?” “What did I have to offer?” “If the world did not use my academic achievements as a criteria for my competence, would I consider myself competent?” “What is self-empowerment?”

As if these questions weren’t maddening enough, I was then tasked to create action plans for my vision of helping communities succeed. To not only think about what I could do to help my community, but to implement my plan strategically. I was charged to act. After eventually organizing student volunteers to help a local farmer meet his annual quota of corn harvest for the year, I left the program invigorated. Years later, after receiving my Masters in International Education Development, I find myself watching debates around the post-2015 agenda and asking again, “What do I have to offer?” (as a self-described member of the youth demographic) and “What can I do?.”

The aim of this post is simply a think piece directed towards policy makers, foundation leaders, civil society members and corporation heads. My question is how can we, with respect to education initiatives, think more broadly about incorporating youth engagement in our policy endeavors in the post-2015 debates? Moreover, can we maximize partnerships with youth, to educate and simultaneously encourage these future economic, social leaders to act towards resolving key development issues?

Encouraging Milestones Towards Acknowledging Youth Perspectives

National and international bodies are taking significant steps to engage as well as understand the scope of youth contribution. The Global Education First Initiative (GEFI), for example, has set a precedent among the international community to refine the commitments made to realize equitable and quality based systems of education for children globally. With reference to youth, GEFI makes notable efforts in partnering with youth advocacy groups internationally to gauge their perspectives on relevant development initiatives. To this end, it was personally interesting when looking at youth (ages 16-30) responses in South Asia, to see that the top two preferred intervention areas seem to be 1) a good education and 2) an honest and responsive government. Mainly because education is vital in developing key temperaments for sustainable development and responsive partnerships with governments (and in general) seem like logical levers for substantial social change. It is encouraging that youth are independently identifying these areas.

In parallel, the United Nations Secretary General, Ban-Ki-Moon, has illustrated his commitment to youth by recently appointing a United Nations Envoy for Youth, Ahmad Alhendawi. Mr. Alhendawi, at 29 years of age is the youngest envoy in the history of the Organization. This move was unprecedented.

On national levels, there are also significant strides being taken to incorporate youth engagement. Taking India as a case study in South Asia, according to the National Consultation Report—Post-2015 Development Framework, according to the 2011 census, an estimated 358 million people (or approximately 30% of the population) are between the ages of 10-24. In February 2014, as part of the Development Framework’s study to understand the population more comprehensively, four NGOs conducted regional consultations with over 100 youth organizations in India. The qualitative findings showed: “The young people want to be an integral part of the process of deciding from planning to execution. Therefore the demand is for an investment with and in young people to ensure that they are able to realize their full human potential as leaders and decision makers/change agents today and tomorrow.” Recommendations from the Framework suggest multiple options from Ministry restructuring to include a more definitive division of Ministry of Youth from Sport, youth commissions, entrepraneurship workshops, media engagement, a focus on youth in fragile states, and legal aid for youth.

Additionally, Ms. Frederika Meijer, UNFPA Representative from India and Bhutan, has developed an interactive Youth of India Portal for various stakeholders in society to check in on vital statistics related to youth demographics. In general, a methodical attempt to track and assess hard data related to Indian youth is at minimum a solid start in researching youth trends.

Looking Ahead

While this is encouraging, the task of formally recognizing these youth partnerships as more then isolated endeavors or as segments to simply address mechanically remains to be seen. Undoubtedly, the factors are complex and there is genuine interest among noted professionals in thinking critically about these challenges. However, sustaining the momentum seems to be vital to future development success.

Internationally, sharing lessons learned and developing joint initiatives could be beneficial. Specifically, there is scope for foundations to partner with each other to exchange financial and intellectual resources. Partnerships between South Asian foundations and foreign companies, for example, could be productive. In essence, collaboration could generate powerful results. Domestic organizations might have the knowledge on what has worked best for their communities, why not allow those organizations to take the lead on tailoring youth specific plans? Or at a minimum, why not invite South Asian Foundations/ Entrepreneurs to the table when it comes to dialogue about future initiatives?

As noted by the earlier mentioned Indian National Consultation Report, there seems to also be untapped potential in forming partnerships domestically between civil society organizations, corporations and international bodies. Outcomes, for example, could be: entrepreneurship workshops, skills in strategic planning, or even earmarking education funding to understand how to interweave this focii into curriculum development. If agencies in the U.S., with a less numerous youth demographic, started incorporating similar initiatives on a state by state basis, the scope for sustainable development initiatives with youth could be extremely positive. To this end, it seems paramount that especially when considering the next steps of designing the post-2015 agenda, we address the need to psychologically prepare youth for development challenges through our commitments.

Further, governments can tangibly benefit by engaging in an intrinsic shift in their value system. Mainly, towards one that recognizes youth as potential architects of solutions to problems, rather than as passive constituents of society. Correspondingly, it seems logical that the subsequent government action plans should honor these commitments and incorporate youth as leaders in the quest to sustain relevant initiatives.

Nayantara Naik is a recent M.A. Graduate from Teachers College, Columbia University. Her M.A. focused on International Education Development, Human Rights and Public Policy. She is currently working on a research team for the Global Partnerships Forum’s book project and is an independent International Education Consultant currently located in New York. Email:

Posted in Uncategorized | Tagged , , | 1 Comment