Report on a Study of the Arrowsmith Program for Learning Disabilities
Prepared by William Lancee, Ph.D., January 22, 2003

This is an abridged version of the above research study. A detailed report with statistical tables is available on request from Arrowsmith School.

Introduction:

Learning Disabilities (LD) seriously affect academic and emotional development and are unlikely to remit without specialized intervention. Students with learning disabilities tend to fall farther and farther behind their peers in academic performance and subsequently tend to have a low sense of self-worth. Klein and Mannuza (2000)1 followed 104 children with LD who initially did not have emotional difficulties. Sixteen years later, these children, when compared to 124 controls, had a much lower status occupational level and continued to struggle with a high prevalence of psychiatric and addiction disorders.

Various special education programs have been developed to address learning disabilities. The approach of the Arrowsmith Program is first to distinguish finely between elemental cognitive impairments and then to implement an individualized highly task-oriented program that exercises and challenges the identified deficit. It is thought that these highly targeted exercises create ways for the brain to provide the necessary functionality for encoding and decoding spoken and written discourse, and for storing, organizing, and integrating knowledge. If this is successful, the child can rejoin his or her peers in normal academic progress. It should be understood that successful graduates of the Arrowsmith Program will require some time to make up for the learning time that was lost due to the original impairment. The authors of the Arrowsmith Program have high expectations for their successful graduates and believe that they will become academically and occupationally competitive.

Objective:

At the beginning of 2001, the Toronto Catholic District School Board (TCDSB) enrolled 30 students (grade 2 to grade 7, from 4 schools) in the Arrowsmith Program (AP). These students were identified by the TCDSB as having learning disabilities. All 30 students were below the age-adjusted 33%-tile in at least one of the three subtests of the Wide Range Achievement Test 3 (WRAT3) - (i) spelling, (ii) timed arithmetic, and (iii) word recognition. Twenty seven students scored below 15%-tile in at least one of these tasks - that is, lower than 85% of other students at the same age. An additional 10 learning disabled students from a fifth TCDSB school were assessed over the school year but were not enrolled in the AP. It was the intention that these students would function as a comparison group.

In October, 2002, Arrowsmith School commissioned William Lancee, Ph.D., who is an experienced research scientist at the University of Toronto, Faculty of Medicine, to review, analyse and report on data from the TCDSB/AP study. The data entry, data verification, data organization, and defining statistical transformations were completed by the middle of December, 2002 when data analysis commenced.

Study Design:

The study design was limited by ethical and practical constraints. It was decided that all students from a given class should receive the same intervention. Therefore it was not possible to select AP students on a random basis. Whole classes could have been randomized to receive AP or not, but this would have seriously increased the required sample size and would have been too costly.

Without randomized controls, it would not be possible to definitively attribute differential improvements to the AP program, since the following selection biases may occur.

(1) It might be that AP students were more amenable to spontaneous improvement or other non-AP factors. However, spontaneous improvement in LD is highly unlikely, especially in a single school year.

(2) The possibility that other non-specific factors such as attention and time could have favoured the AP students is also unlikely, since comparison students received similar added attention and learning time.

(3) It could be argued that if students who were selected for AP had less severe LD, they would be more likely to improve. As will be shown in the result section, this possibility could also be rejected, since initial severity was not predictive of degree of improvement.

The approach taken was to test for pre-post improvements in the 30 AP students, and then test whether this was different from the pre-post changes in the 10 comparison students. Because of the small sample sizes, the power to detect statistically significant difference between the two groups was low. Nevertheless, if statistical differences were found, they could be accepted with confidence.

Study Sample:

There were no study dropouts, and all 30 AP students and 10 comparison students were followed up. Because the basis for selection was LD, there was no control over grade and gender.

Although all study students had LD, no single definition was used to select students for the study. There were three comparison students who were close to normal range on at least one WRAT3 subtest (75%-tile or higher). None of the 30 AP were functioning close to this level. This lack of equivalence at the start made direct comparison of the two groups difficult.

At the risk of reducing the size of comparison sample too much, the three highest functioning comparison students were considered to be outliers and were treated as a separate subgroup for some of the statistical analyses.

Measures:

Pre and post measures include the following 12 standardized Achievement and IQ tests:

• the Wide Range Achievement Test 3 (WRAT3)

1 spelling
2 timed arithmetic
3 word recognition

• the Woodcock Reading Mastery

1 word identification
2 word attack
3 word comprehension
4 passage comprehension

• the Monroe-Sherman Achievement Tests

1 copying text
2 auditory memory
3 visual memory

• the Otis-Lennon Mental Ability IQ Test (timed task)

• the Peabody Picture Vocabulary IQ Test (task is not timed)

Eighty of the 960 test administrations were done jointly by two different test administrators (one designated by the TCDSB and one by the AP). These score pairs correlated highly (r=.90). In all but two cases the tests scores were identical, indicating excellent adherence to test protocol.

AP students are assigned exercises based on specific cognitive deficits. Therefore the Arrowsmith Program uses assessment tools (AP authored) to test performance on elemental cognitive functions. These tests were also done at pre and post by the 30 AP students and by the 10 comparison students. The AP tests were used to determine if improvements in standardized tests could be linked to improvements in the hypothesized elemental cognitive functions.

Students in the AP and their teachers and parents completed a comprehensive satisfaction questionnaire at the 12-month follow-up time point.

Relationship between improvements and satisfaction

The 30 AP students, their parents and teachers completed a 24 item satisfaction questionnaire. Improvements were seen by at least 2 raters (teacher and student; student and parent; or teacher and parent) in more than 80% of students in the following areas: reading comprehension; ability to focus on task; understanding ideas; legibility of written work; confidence; self-esteem; and ability to self-advocate. Between 70% and 80% of students were seen as having improved in: telling time; remembering factual information; listening skills; organizational skills; and understanding and following instructions.

The correlation between improved comprehension as seen by teachers correlated highly with the Relative Progress GE scores (Pearson r = 0.49; p<0.01).

Association between improvements in Achievement/IQ tests and improvements in AP tests

If improvements were due to specific AP exercises then we should expect to see strong correlations between changes in AP test scores and changes in Achievement/IQ test scores. The results support the defining principle of the AP program: that carefully targeted exercises aimed at improving elemental cognitive abilities will result in broader improvements seen as increased scores on achievement and IQ tests.

Conclusion

Despite some study design limitations and small sample size, the study results strongly support the Arrowsmith Program as instrumental in changing the developmental course of the majority of children with LD in this sample. In only 12 months, almost one third of the AP students were on a course that brought them closer to their peers. Another 27% improved their performance at the same rate as expected from their non-LD peers, that is, they stayed at the same distance but did not fall further behind. All other AP students (43%) improved at least somewhat on the various achievement tests. None of the 10 students in the comparison group progressed substantially beyond their entry status.

1Klein, R.G. and Mannuza, S. (2000). Children with complicated reading disorders grown up. In L.L. Greenhill (Ed.), Learning disabilities: Implications for psychiatric treatment. Washington: American Psychiatric Press.

Study Consultant

Dr. William Lancee is Head of Research in the Department of Psychiatry at Mount Sinai Hospital and Associate Professor, Department of Psychiatry, University of Toronto. He has a Bachelor in Mathematics from the University of Waterloo. In the first part of his career, he was a statistical consultant to major pharmaceutical companies. Later he received his Ph.D. from the Institute of Medical Science at University of Toronto, and has designed and carried out more than 20 peer reviewed collaborative studies, including large-scale epidemiological studies as well as psychotherapy intervention studies. He is author of three published psychological measurement tools: The Nurse Observed Behaviour Scale; the Staff Patient Interaction Response Scale; and the Perceived Family Burden Scale. He has supervised 15 graduate students. His special interest is in the mathematical modeling of complex systems using cellular automata, genetic programming, and neural networks. He has published 37 papers on a wide variety of topics. Recent papers related to family issues and child development are:

Maunder R., Lancee, W.J., Greenberg, G., Hunter, J., Fernandes, B. Insecure attachment in a subgroup of ulcerative colitis defined by ANCA status. Dig Dis Sci, 45, 2000, 2127-2132.

Hazelton, R. Lancee, W.J., O'Neil, M.K. (1998) The controversial long-term effects of parental divorce: the role of early attachment. Journal of Divorce and Remarriage, 29(1)1-18.

Beitchman, J.H., Brownlie, E.B., Inglis, A., Wild, J., Ferguson, B., Schachter, D., Lancee, W.J., Matthews, R., Wilson, B. (1996) Seven-Year Follow-up of Speech/Language Impaired and Control Children: Psychiatric Outcome. Journal of Child Psychology and Psychiatry 37(8)961-970.

Levene, J.E., Lancee, W.J., Seeman, M.V. (1996) The Perceived Family Burden Scale: Measurement and Validation. Schizophrenia Research, 22, 151-157.

Beiser, M., Lancee, W.J., Gotowiec, A, Sack, W., & Redshirt, R. (1993). Measuring Self-Perceived Role Competence Among First Nations and non-Native Children. Canadian Journal of Psychiatry. Vol 38, 412-419.