We conducted oral assessments at all the Asha supported schools in Tamilnadu and UP in Nov 2023. More than 180 schools and 15000 students from classes 1 to 8 were covered in this assessments.
Oral Assessments at Asha Chennai Projects (Nov 2023)
Background
Asha Chennai has been conducting assessments at all the schools that we support every year ever since 2014! These assessments are based on Pratham’s ASER toolkit.
At Asha Chennai we are supporting 160+ government schools spread across Tamilnadu. Oral assessments provide us the metrics to tell us how our projects are doing and start measuring the impact we are having and fine-tune our actions so as to improve the outcome.
Here are photos from the Oral assmts.
Assessments over the years
Asha Chennai started conducting assessments in April 2015. In that 2014-15 academic year as well as 2015-16 academic year we conducted primarily a written assessment and conducted oral assessment only for weaker children. For children who were at grade level we conducted only the written assessments. The first oral assessments were conducted in Nov 2016 in the 2016-17 academic year. The following graph shows the number of schools where the assessments were conducted in the various years.
Both the number of schools and the number of children covered have significantly increased over the years. The big dip in the years 2020-21 and 2021-22 was because the schools were closed and the numbers here reflect the mini-schools we were running. In 2021-22 we conducted oral assessments in Oct 2021 at our mini-schools and then once again in the actual schools later in April 2022. The huge increase in 2022-23 and 2023-24 are partly because of our growth. But these also now include 8 and 12 schools resp. in UP. Further we have also started conducting these assessments for students in classes 6 to 8. This has also resulted in the increase in the numbers.
Oral Assessments 2023-24
Asha Chennai conducted the Oral assessments at all the supported government schools. The schools covered are listed below.
Project Sangamam (Thiruvallur/Kanchipuram Dists) – 91 Schools.
Project Pearl (Thoothukudi/Thirunelveli/Then Kasi Dists) – 28 Schools.
Project Thulasi (Viluppuram/Thiruvannamalai Dists) – 25 Schools.
Project Poorna Vidhya (Chennai/Chengalpet Dists) – 9 Schools.
Project Manigal (Chennai Dist) – 2 schools.
Project Cauvery (Thiruvarur Dist) – 12 schools.
Project Sangamam Rajatalab (Varanasi Dist of UP) – 11 schools
Total – 178 Schools.
Like last year this year too the teachers at every project were trained personally by Rajaram to ensure consistency. Over the years we have evolved a detailed set of instructions that makes the process as objective as possible. This was shared with all the teachers. The assessments were also carefully monitored by volunteers and lead teachers.
Sangamam – Thiruvallur/Kanchipuram
Thiruvallur teachers were trained on Oct 28th. The computer teachers of the project conducted the assessments. They were also assisted by the CS trainers and KaradiPath teachers where required. They were teamed up teams of 2 or 3 teachers. The assessments were conducted from Nov 6th to 15th at the Thiruvallur Schools.
Pearl – Thoothukudi/Thirunelveli
None of the volunteers could travel personally to train the teachers. Online training was held for them along with Thulasi teachers on 28 Oct,2023. The assessments were then conducted there from 30 Oct-2 Nov, 2023.
Thulasi – Viluppuram / Thiruvannamalai
The training for the Thulasi teachers also took place online on 28 Oct 2023. The assessments at all the schools were conducted from the Nov 6-15.
Poorna Vidhya/Manigal – Chennai/Kanchipuram
Rajaram trained the teachers in person on 10th Nov 2023 at Chennai. The Manigal teachers also joined the Poorna Vidhya teachers for the training. The assessments at all the schools were held from the 16th to the 26th.
Cauvery – Thiruvarur
As Rajaram was travelling to visit the Thiruvarur project along with the steward Komalavalli on the 3rd and 4th of Nov, we decided to start the assessment at that time with an in-person training. The training was held on Nov 4th at PUPS Nerunjanakudi. Rajaram trained the teacher on how to conduct the assessment. Then the teachers conducted the assessment for the children at that school. Rajaram observed the assessment and gave feedback to the teachers. They then conducted the rest of the assessments from Nov 4 -18.
Sangamam Rajatalab – Varanasi Dist, UP
The oral assessments are conducted in Hindi, English and Maths at our UP schools. The training also needed to be conducted in Hindi and English. Rajaram did the training over a Google Meet session along with the project steward Komalavalli. The teachers then conducted the assessment at their schools from 17 Nov to 5 Dec, 2023. This year the assessments were held for middle school students as well. Therefore, the coordinator also needed to talk to the HMs and get the permissions.
Data Collection
Asha has been gathering data about the school and the students along with the information on the performance of the students. There is often wide variation in the performance between various schools that we support. While standard measures like teacher-student ratio, motivated vs unmotivated teachers, richer vs poorer area, etc. account for some of the differences, there is still a lot of differences that still need explanation. We decided to collect data to correlate against the performance of the children and the schools. Here are the data that we are collecting now.
School Data
We got the school level data from DISE (www.schoolreportcards.in). The data included the following,
- Strength of the school (which we also had from our own data) and the trend of the strength over the last few years.
- No of male and female teachers. These two together give the teacher-student ratio.
- The percentage of SC and ST students. This to a large extent also reflects on how poor and remote a place is.
- The percentage of BPL families.
- The strength of the school the last two years which gives us an idea of whether the school is gaining or losing strength.
- If the school a middle school or primary school?
Student Data
In addition to the school level data, we also gather data are the level of an individual student. We collect the following student level data:
- Gender – boy or girl – We were collecting this even earlier.
- Student’s pre-school education – Did he attend a Balwadi, a Kindergarten (if so for one or two years) or none?
- Parent’s educations. What is the education level of the mother and father?
- Date of Birth which will tell us the student’s age.
- Height and weight. This will enable us to check if the children is at a correct height and weight for the age/class.
- Attendance percentage for the student.
- Distance of the student’s home from the school. We are not exactly measuring the distance. We classify the student as a. Coming from the same hamlet as the school, b. Coming from a neighbouring hamlet by walk or c. Coming from a more distant hamlet by vehicle (usual share auto or bus).
- Is the student going for tuition?
- Is the student regular with his/her homework?
Result Analysis
More detailed correlation with all the data collected will be done once the written assessment is also completed. In the meantime, here is some basic information that we are able to immediately see from the data.
Recovery of Education Post-Corona
Here are three graphs showing comparision of the result in TN beween this year and last. Thick line represents this year and dashed line last year.
As you can see there has been a good improvement in English and Tamil and no major change in Maths. We decided to compare the performance against the data from 2019-20, the last year before Corona.
Note that in 2019-20 we conducted the oral assessments only for primary school children. Here again it looks like we have caught up with the pre-Corona performance in English and Tamil. However, our Maths performance is still lagging. Part of the change may also be related to the fact that we are following the Ennum Ezhuthum (Foundational Literacy and Numeracy) focused textbooks and teaching pattern.
Effectiveness of KaradiPath
In 10 schools we started the KaradiPath program in Oct 2022 and the children in these schools are continuing the level 2 of the KaradiPath program. In 27 more schools we started the level 1 of the KaradiPath program this year. The first set of graphs, we are comparing the performance of the KaradiPath Level 1 schools with all TN schools. Thick line represents KaradiPath L2 schools and the dashed lines all schools in TN.
As you can see there is an outperformancein English but not in Tamil and Maths. Here is the comparision of the English performance of these same schools in 2019-20.
As you can see there is much less of an outperformance in 2019-20 by these schools. But the odd thing is that there is an outperformance even by the 6th to 8th classes in these schools. Our KaradiPath program should only have an impact on the primary students as we are doing it only for classes 2 to 5!!! Therefore this requires a little more analysis.
Here is the performance in English of the level 1 KaradiPath schools in 2023-24 and then in 2019-20.
Here the numbers are a little clearer. There is a clear outperformance in 2023-24 vs. 2019-20. And the outperformance seems focused more in the primary classes like we would expect.
The aberration of the middle school outperformance may just be due to the small number of middle schools in the first group (2 of the 10 schools). These schools may have had other things happening (like say a change in the English teacher or something like that).
On the whole we are glad to note that KaradiPath program is indeed impacting the English performance positively while not negatively impacting the other subject performances (i.e. by taking away time from them).
UP vs TN Schools
In the analysis of the assessments from 2021-22, we had observed how the UP school children were doing much better in Hindi vs. the TN students in Tamil. But there is an equivalent outperformance of the TN students in English. Maths performance was more or less similar. Here is that report. During our site visits, we were also able to observe that this dramatic outperformance in mother tongue was indeed there.
In this year’s assessment, we are seeing that this outperformance has increased further and has now taken place in Maths as well. Even in English the gap has reduced significantly. Given the significant improvement in all subjects, we are also suspecting that the assessments may have been conducted a little more liberally by our teachers in UP this year. We will continue cross checking these results. We will be conducting written assessments as well in UP this year. This will give us a way to cross check our results. In the mean time, here is a comparision of the performance of UP (thick likes) and TN (dashed lines) in Engliah, Maths and Tamil/Hindi.