[MLE]"What's Wrong With Our Teaching?"

Dear MLE Interest Group,

India Today had recently some interesting topics on education in India. In October there was a rather critical report on SSA. Last week they had an item on education in the metros. Even though it does not touch on language I think many of you would be interested in the report on which this article was based.  You can download it from http://www.indiatodaygroup.com/itoday/20061127/  I copy an executive summary below.

Shalom,

Karsten

Karsten van Riezen
Education Consultant, SIL Int.

Director
SIL, South Asia Group.

India today article:

What's Wrong With Our Teaching?

An exclusive survey of India’s top schools exposes alarming gaps in student learning, with performance falling way below international levels.

STUDENT Learning in the Metros 2006

Public debate in India bemoans the lot of government schools in the country. The implicit assumption is that all’s well (or at least almost well) with private schools. In this research study, an attempt has been made to verify that assumption. And instead of looking at private schools in general (which come in a wide variety), an attempt has been made to measure how well students are learning in the ‘top’ English medium schools (as per public perception) in 5 metros - Mumbai, Kolkata, New Delhi, Chennai and Bangalore. Over 32,000 students of classes 4, 6 and 8 participated in this study. An analysis of their performance suggests that even in our ‘top’ schools, students are not learning well and with understanding. Schools seem to be laying disproportionate emphasis on rote and procedural learning and not surprisingly, students tend to be strong in those. To a certain extent this is good - for one, it builds habits of rigour and hard work. But when it starts replacing original thinking and creativity, over-reliance on rote can be extremely counter-productive. This is happening, and we need to be alarmed.

DESIGN OF THE STUDY

An expert panel of educationists and principals guided the survey. About 200 people from different walks of life were surveyed in each metro to identify the best schools in the country. Based on their responses, a list of 50 top schools was drawn for each city. These schools were then invited to participate in the study. Students of classes 4, 6 and 8 of each school were tested for their learning achievement with a special test tailored to their age and ability. The test tried to measure how well students of these classes understand the key concepts in English, Mathematics and Science. Apart from the multiple-choice questions, students were also required to write a small paragraph or essay, which would help study their writing competencies. A secondary study was also conducted to understand the progression of learning achievement across the classes. In this study, a common test was administered to students of classes 4, 6 and 8, to gather insights on the retention and development of knowledge, as students move to higher classes. Additionally, about 25% of the questions in this paper were taken from an international assessment study (the Trends in International Maths and Science Studies - TIMSS, http:// www.timss.org) for which performance data of students from over 40 countries is available.

TEST ADMINISTRATION AND DATA ANALYSIS About 32,000 students from 142 schools participated in the tests which were conducted between February and April 2006. Trained invigilators from Educational Initiatives supervised every test. The participation ranged from 23 schools in Kolkata to 37 schools in Chennai. Some background information on issues like class size, fees and school facilities were also collected through a questionnaire from the participating schools to look for any influence of these factors on student achievement. 89 out of the 142 schools filled and returned the background questionnaire.

THE FINDINGS

The results do not present a happy picture of the state of student learning even in the ‘top’ schools of the metros. Students seem to be learning mechanically, and are able to answer questions based on recall or standard procedures quite well. However, their performance on questions testing understanding or application is far below what we consider to be acceptable levels. The student performance suggests that they are unable to tackle questions that appear to be a little different from what they typically find in textbooks or in the class. Their ability to apply what they have learnt to new, unfamiliar problems - so important in today’s world - is not very high. The results also show that students tend to slot learning into artificial compartments. They may learn something, but are able to answer it only in the same context, in which the learning first occurred. They may be using an aspect of what they have learnt in their day-to-day lives, but be completely unaware of that connection. Another finding is that students tend to be weak in certain real-life competencies like practical measurements and problem solving, which can and should be developed through the formal school curriculum.

Many of these findings were corroborated through the secondary study in which learning levels across classes were compared. While learning clearly improved from class 4 to 6 to 8, a number of students seem to be learning class 3 and 4 concepts only around class 6 or later.

A number of specific misconceptions in English, Mathematics and Science were also identified, and these are illustrated with a large number of examples in the detailed report.

COMPARATIVE ANALYSIS

One of the most significant findings of the study was the poor performance of the students compared with the average performance of students from 43 countries. Across the sample of 11 questions in Maths and Science, our class 4 students performed below international average on all of them (Exhibit 2).

A comparative analysis of the performance of the 5 metros again threw up a surprise. It was found that the performance of the cities fell into two categories, with Kolkata, Mumbai and Delhi clearly outperforming Bangalore and Chennai (Exhibit 1). It was also found that schools affiliated to the CISCE (ICSE) board out-performed the CBSE board which in turn out-performed the state boards.

Boys outperformed girls by a margin that was statistically significant in Mathematics (in all classes) and Science in class 8. In Mathematics, the gap widens even more among better-performing students and in the case of difficult questions. We believe that these differences are not because boys are inherently better in Mathematics than girls, but due to social messages encouraging boys to do better in Mathematics and probably discouraging girls.

As stated earlier the exhibit 2 shows that the average performance of the student in the metro schools was below the international average on all the common questions taken from the TIMSS.

None of the factors like class size or school facilities seemed to be strongly and clearly correlated to the student performances in the tests. Our hypothesis is that the teaching-learning processes and the quality of leadership play an important part in determining the effectiveness of student learning.

NEXT STEPS

All the data, including the question papers and detailed analysis will be made available in the public domain. This should allow the issue of quality of learning to be more widely debated on a foundation of hard data, rather than subjective “opinions”. It is planned to expand this study to more cities in the coming year, and also enhance the study in other ways. It seems clear that it is in our power to improve the quality of learning in our schools, but that will happen only if we choose to make that commitment to the next generation, by way of focusing on real learning. The current focus on valuing high scores in the board exams or fancy facilities in our schools is unlikely to take us far, as far as real learning is concerned. Our tests should be such that they measure real learning.

Executive Summary v © Educational Initiatives and Wipro Many of the questions asked in the study test understanding of concepts covered in textbooks and classrooms of a lower class. Of course, the questions are of a form that are slightly different from what is typically done in the class. In many such questions, it was found that basic understanding seemed to be weak among a significant proportion of the students.