Written assessments in English and Maths were conducted at all the Asha Chennai supported schools from Feb 14th to March 10th. It was conducted in 55 schools covering over 3300 children. The papers have been graded and reports generated for all the schools (and dynamically viewable from our server). Click here to see all the photos in our Assessment photo album.

Background

Please read the earlier report on the Oral assessments that were conducted in Nov 2016 : https://ashanet.org/blog/2017/01/asha-chennai-assessments/. This describes all the efforts undertaken by Asha Chennai leading up to the Oral Assessments of Nov 2016. This also describes our goals for these assessments (both oral and written), our learning from the previous assessments that we conducted.

Click here to see all the photos in our Assessment photo album.

Coverage

In Nov 2016 we did the full ASER oral assessments at all the schools to which we send regular teachers as part of Project Sangamam and Project Pearl.

The schools to which we send computer teachers once or twice a week and schools from other projects Poorna Vidhya and Project Thulasi were not covered. The written assessment conducted in Feb / Mar 2017 covered all of the above schools. The schools covered were as follows,

Project Sangamam 14 schools

Project Pearl 3 schools

Project Thulasi 5 schools

Poorna Vidhya + 4 (incl. Olcott Memorial School)

Project Sangamam Kanini 29 schools

Total 55 schools

Total Number of Children 3295

Given absentees etc., the tests were actually administered to 2881 students.

Assessments Feb 2017

Like we did in Feb 2016, we decided to do the following.

1. Conduct written assessments in English for classes 3 to 5 and in Maths for classes 2 to 5.
2. For children who are not given written assessments and for children who are unable to make good progress with the written assessments also give oral assessments. Note this was not done in schools where full oral assessments were already done. We already had the data that was required.

Over the month of January we designed the assessment papers. Maths tests were developed by Mrs. Meena Suresh of Ramanujan Museum and Centre for Math Resources. In Maths, we set the papers at the same level of difficulty as last year. Last year the papers focused on the 4 arithmetic operations. This time the coverage of areas was a little broader.

Our English tests were developed by Mrs. Jayashree Arun, a volunteer of Asha Chennai and works at P.S. Senior Secondary School in Chennai. In English, last year we had found that the papers for 4^th and 5^th standard were very difficult for the children. Therefore we decided to make the papers a little easier. We made the following changes.

• Reading comprehension questions have more direct answers.
• Sentence completion questions were made easier.
• Sentence translation question for 5^th std was replaced by word translation questions.

Conducting the Assessments

The assessments were conducted from Feb 14^th to March 10^th. At Thiruvallur and Chennai (total of 47 schools), we conducted the assessments using the computer teachers from Sangamam Kanini. The advantage was that they were not tied to one school. We trained them on how to conduct the tests. It is very important to conduct the test consistently. The way the teacher answer questions by the children, how and whether they translate / explain a question etc. make a big difference in how the children perform. Some rules we followed are,

1. Teacher can explain the question when asked by a student but should not say anything more than the question stated in the paper. They should not start explaining how the question needs to be answered.
2. For Maths word problems, the teacher may translate to plain language (esp. English medium) for all students.
3. For 2^nd std. Maths, the teacher should walk the children through the paper and make them answer. We didn’t do that in the first day and the children at a few of the schools, struggled a lot.
4. There were also problems with specific questions. The children were not used to the question being asked the way it was. In cases like that the teacher can frame the question in the way the children understand them.
5. The school teachers (incl. the Asha teacher) should ideally not come into the room when the test is being conducted and should certainly not interact with the children.
6. At the end of the test, when the papers are being collected, the teacher may check the paper and if there are some obvious silly mistakes, point them out to the student and give them another chance.
7. Also the teacher collecting the paper needs to check if the name of the student is written legibly and then group the papers into classes and seal them for correction.

Further the teachers should take with them a good number of pencils, erasers, sharpeners, etc. At the end of the test, each student was given 2 chocolates to make them feel good.

The computer teachers were made into 5 pairs. They covered all the schools in Thiruvallur in a span of 8 days – one school a day.

Rajaram later trained the teachers in Kayathar (Project Pearl) over phone and then travelled to Senji to meet with the teachers there as well as personally administed the assessments in one school. In early March all the teachers from Thiruvallur came to Chennai one day to conduct the assessments in all the Chennai schools.

On the whole the assessments were well conducted. There were very few problems in the way they were conducted.

Correcting the Assessments and Entering the Marks

We started correcting the assessments even before they were completed everywhere. Two of the senior computer teachers at Thiruvallur did the corrections for two days a week till everything was completed. They also took the help of two more teachers when they were doing it. They corrected about half the papers. The other half was corrected by a group of volunteers.

Like the administration of the tests, the corrections also needed to be done consistently. Things like, should partial marks be given for a question and if so for what? Should we look for spelling correctness or not? Should we look for full sentence response or is just one word answer enough? Also the people correcting the paper had to keep their eyes open for anything that looks like mass copying or assistance by some adult.

We also created Excel templates for entering the marks. As the papers are corrected these were also entered into the corresponding Excel template for that class/school. These were then processed to load them into the server database. All these were finally made available through our server around the 14^th of April.

Analysis of the Performance

Displays at an overall level, at a school level, class level and at an individual student level have been provided on our server. At the overall level we provide a grid that shows the performance of each school / class / subject. The cell is marked green if the school has performed one sigma above the average, red if it is one sigma below the average and white if it is around mean+-sigma. The overall display also provides a comparison of All students vs Girls vs Boys. The performance of children in each of our projects is also listed.

The school level display shows the attendance and performance of each class / subject. In each class / subject we also show the performance of same class / subject last year. We also indicate if this has shown an improvement or has gone down. The comparison of the oral assessment of this school with all the schools is also provided. See the oral assessment report for more details.

The class level display shows the following:

1. List of students with their marks / oral status in English and Marks. It also compares their oral status against it last year. This should not slip since this is such a basic measure. In very few cases it has slipped. We also list the marks obtained last year by the same student as well.
2. List of strong and weak areas in English and Maths. The strong areas are the up to 5 questions in which this school has most outperformed the all school average for that question. Similarly weak areas are the up to 5 questions where the underperformance has been the most.
3. Graphs showing the percentage of students in different oral assessment status bucket vs. all schools.
4. Graphs showing the performance of the school students in all the questions in English and Maths and comparison of that with all school students.

At an individual student level the teacher can see how the student has performed in each question.

Some of the analysis that are still pending are:

1. Comparision of how classes/sections that are taught by Asha teachers have fared compared to all classes/sections of the schools they go to.
2. Comparision of the performance of the schools against various data available for the school like absentee percentage (we have the data during the days of the oral and written assessment to start with), RTE grade for the school, Teacher student ratio, Percent of SC/ST population (which tends to be poorer) etc.

Learnings from the Assessments

As always when you conduct a massive assessment across such a wide geography of schools, there is bound to be learnings. Here are some basic things we have learnt.

1. In English, despite our making the tests easier for classes 4 and 5, they have still struggled. Learning beyond vocabulary doesn’t seem to be happening.
2. In Maths class 2 children struggle with the written test in many schools. We need to structure a way to conduct the tests. This year the performance of the children in class 2s across schools wouldn’t have been comparable.
3. There are wide variations in the performance of children across all the schools. Some of the important factors that affect performance seem to be: Teacher student ratio, Absenteeism, Percentage of Tribals in the school. The last factor I believe has an influence because of the poverty level of the village, remoteness of it, lack of value for education, migrant nature of the population etc.

On the whole the schools where we are continuing to support have done slightly better than last year. We were hoping for a more significant improvement. We found the many of the same problems as last year. We added a few more pointers to the schools in our report this year. These are,

• While the performance in arithmetic operations, comparision etc. have improved, understand of place-value-system and how numbers are formed needs to be firmed up.
• Teachers can call out the sum and the students should write and do the sums. This will help in doing the work independently. i.e. Teacher should not write the sums in the board or their notebook all the time. This will help children get their place values right.
• Please revise earlier class and term topics. Many of the students (from all schools) failed to answer easy questions that would have been covered in earlier classes.
• Basic arithmetic operations can be provided to the children in multiple formats. Children often get things wrong if the sum is given in a format they are not used to.

Some of the suggestions we have made for English are,

• Make them listen to the read out of the lesson provided in the computers. Help them follow what is being read out in the textbook by running their finger over the words. Get the whole class to read aloud the lesson after the teacher (or the recording).
• Teach them basics of phonics. You may use materials like the StarFall that we have provided in the computers. The system used in some of the schools trained by the SSA also works well.
• Please give random dictation words to the children (oral and written). Also ask children questions with one-word answers from the lessons.

To each school we have provided a report with a cover letter stating the above points, the school level display and class level display for each of the 5 classes. Note the school level and class level displays were described in the previous section.