SuccessMaker? Any good?

Replies to This Discussion

Permalink Reply by Miguel Guhlin on March 8, 2009 at 5:48pm

I encourage you to review this research...
http://mguhlin.wikispaces.com/Integrated+Learning+Systems+(ILS)+Research

• In a review of 100 studies of ILSes, Henry Jay Becker found that they “provided little evidence of ILS impact on student achievement. Where differences were found between the achievement of ILS users and comparable non-users, Becker concluded they were too small to have any educational significance.(Becker, 1992).
• Most scientific studies of ILSs have failed to demonstrate their impact on student outcomes (Van Dusen, Worthen, 1994).
• Skinneran operant conditioning learning theory provided the basis for ILS development (Mazyck, 2002). Instructional technologies must be optimized to support the cognitive requirements at hand, rather than only focusing on the behavioral model of learning theory. (Hooper et al., 1991)
• Independent studies of integrated learning system technologies have subsequently confirmed that learning discrete skills in isolation does little to support students in transferring knowledge to other domains of experience. This lack of transferability of skills from integrated learning system performance to other tasks is well-documented in the research literature (NCREL, 2002).
• Educators question the many evaluations of ILSs conducted by the ILS companies themselves. Most of the studies of ILSs were inconclusive and had many serious flaws (Mazyck, 2002).
• When ILSes are uses as adjunct activities or when teachers don’t chart progress and use results to complement curricular efforts, they have no positive effect on student achievement or attitudes (NCREL, 2002).
• Integrated learning systems have a long way to go before they can receive an unqualified endorsement (BECTa 1998, having spent $1.3 million on an extensive review of ILS vendors including Successmaker, Plato, and others).
• Although teachers generally like Successmaker as a system for enhancing basic skills, in most cases this was due to a general feeling about its positive or potential benefits rather than direct evidence.
• Students using Successmaker for more than a term had become less enthusiastic about the system.
• Successmaker won’t work if you use it for less than 3 times a week and just leave it to “get on with things on its own.” (Ian Hedley, Carter Community School, Successmaker Coordinator).
• ILS should abandon their “mindless adherence to the principle of individualized instruction.” (Becker, 1994)
• The Successmaker results relate to how they were publicized. Studies in reports from the vendor all show substantial positive effect sizes. Those from independent sources show modest or negligible effects (Bracey, 1991).
• Costs of ILSes are too great given its record of effectiveness (Levin, Destner, & Meister, 1986).
• Machines are not as effective as live teachers; ILS teaching is too mechanical, too impersonal; Pupils will find ILS instruction boring and repetitive, and thus can
lose their motivation to learn; ILS can teach routine skills but they cannot teacher higher order thinking skills or conceptual thinking (White, 1993).
• Computer based instruction is consistently more effective than traditional instruction, but the amount of improvement is low to moderate (Balajthy, 1989).
Selection
1. A team comprised of the participants—teachers, community members, administrators, students--should be in on the ILS selection (Hill, 1993).
2. Software and ILSes should be chosen by classroom teachers. Once selected, the system should be integrated into the local curriculum by the appropriate district curriculum staff. This role should not be usurped by computer teachers or the district computer coordinator or director (Wiberg, March 1992).
3. Schools looking at ILSes should not look at a company’s curriculum correlation charts, but sit down and look at the correlated lessons in-dept. Your curriculum has to be the baseline (Mageau, 1992).
Implementation & Staff
1. Hire a competent ILS systems manager. Most teachers need a support person who is the in-house ILS expert. These managers should be teachers who are knowledgeable about the curriculum and technical aspects of the system (Sherry, March 1992).
2. Plan site visits to talk about implementation, focus, funding, ongoing support and other relevant topics (Hill, 1993).
3. Comprehensive training for teachers and administrators on the ILS should begin PRIOR to its arrival and done by ILS consultants. In this way, the district can avoid the problem resulting from the lack of teacher support (Hill, 1993).
4. Get teachers to go to the lab with their students and circulate among them while they’re online. (Mageau, 1992).
5. The services of a SuccessMaker technician (lab manager) is desirable. This person is able to print out the reports and worksheets as requested by staff and able to provide necessary in-house support for teachers (Smith, 2002).
Professional Development
1. Give teachers time to preview lessons. Teachers should be given additional in-school time specifically for previewing ILS lessons. ILS curriculums are extensive; they can’t be taken home like other materials. Teachers should also have access to the ILS lab during after-school or weekend hours (Sherry, March 1992).
2. Most teachers never get the level of training necessary to use most of the ILS’s features (Sherry, 1992).
3. Teacher who is more involved with the ILS work of her students will likely produce the greatest learning gains
4. Teacher who had far and away the most successful effect size [substantial increase in test scores] knew the most about the system, knew the most about what the kids were doing in the lab, and went back to the classroom and made
decisions about what to do based on that information (Mageau, 1992).Use cooperative learning approaches to enhance the effectiveness of ILSs with diverse groups of students (Becker, 1992a; Brush, 1997a; Brush, 1997b; Brush, 1998; Hooper & Hannafin, 1991; Hooper, Temiyakam & Williams, 1993;Mevarech, 1994 as cited in Mazyck, 2002).
5. Cooperative learning paired with ILSs produces more willingness to take on difficult tasks and persist, long term retention of what is learned, higher level reasoning and metacognitive thought, creative thinking, transfer of learning, positive attitudes towards the task being completed and greater time on task. It also improves student achievement and attitudes in computer-based environments.
6. When pairs of students work cooperatively to complete exercises in an ILS, they OUTPERFORM their counterparts who use the system on an individual basis (NCREL, 2002).
7. Effort must be made to facilitate students’ transfer of knowledge to other domains of experience (NCREL, 1996). “Students may learn isolated skills and tools but they will still lack an understanding of how those various skills fit together to solve problems and complete tasks.”
8. Should be an integral part of the district’s educational strategy and guide professional development of teachers (Tingey, 2002).
Target Population
1. Students may be off-task if they are not supervised in an ILS implementation (this research referred specifically to Successmaker). (Underwood et al, 2002).
2. Some conclusions on computer-based instruction (CBI):
a. The lower the grade level or ability of the students, the more effective CBI is
b. CBI is consistently more effective than traditional instruction but the amount of improvement is low to moderate
c. Structured CBI, with emphasis on direct instruction, is more effect in producing achievement gains than unstructured CBI
3. The majority of computer at the secondary level was for introductions to the computer and computer programming. The secondary usage of computers was for low-level drill and practice with the typical student receiving such instruction only 15-20 minutes each week, hardly enough to gain proficiency in any skill being learned.
4. Low socio-economic status (SES) schools with predominately minority populations used their computers to administer drill and practice computer-based instruction.
5. Low SES schools with predominately white populations preferred to use computers with their higher-achieving students to teach programming and computer skills (Balajthy, 1989).
Reading Research
1. Computer materials are most effective when they allow the learner some degree of decision-making and control over the task.
2. Interactive materials are imperative.
3. When possible, learners should be able to self-check by looking back at previous work or calling up help features.
4. Preference should be given to programs that require the reader to make decisions about the progression or direction of the content, thus, promoting active involvement.
5. When viewing reading as a process of constructing meaning, the implications for computer-based instruction or the use of ILSs emerge as (Strickland, Feeley, and Webner, 1987):
a. Programs focusing solely on isolated skills, such as word recognition and word meanings are apt to be more effective when these skills are offered in context and in conjunction with one another. Isolating skills often fosters habits that are ineffective in authentic reading situations where understanding the message is essential.
b. CBI should be meaningful the reader, and it is unlikely that students will learn very much about reading from programs that defy all efforts to make sense of them.
References
Balajthy, E. (1989). Computers and Reading. New Jersey: Prentice Hall
Becker, H. J. (November, 1988). The impact of computers on children’s learning. Principal.
Bracey, G. (September, 1991). ILS research isn’t helpful. Electronic Learning 11 (1), p.16
Hill, M. (January, 1993). What’s new in ILSes? Electronic Learning, 12 (4), pg.12.
Hooper, S. & Hannafin, M. (1991) The effects of group composition on achievement, interaction, and learning efficiency during computer-based cooperative instruction. Educational Leadership.
King, J. B. (December, 2000). Use of integrated learning systems to enhance the proficiency of poor readers in middle school. Unpublished manuscript.
Mageau, T. (January, 1992). Integrating an ILS: Two teaching methods that work. Electronic Learning, 11 (4), pg.16
Mazyck, M. (March/April, 2002). Integrated Learning systems and students of color: Two decades of use in K-12 education. TechTrends 46(2), pg.33
O’Neal, S. (June, 2002). The impact of ESEA’s scientifically based research requirement on schools’ technology solutions. T. H. E. Journal, pg.64
Sherry, M. (March, 1992). Integrated Learning Systems: The ILS Mac Pack. Electronic Learning, 11 (6), pg.10; New York.
Smith, J. H. (1997). TCT ILS Project (Successmaker) Conclusions. Available online at http://atschool.eduweb.co.uk/gyhsmith/itnet/jhsmith1.htm on 7/1/2002.
Strickley, D. S., Feeley, J. T. & Webner, S. B. (1987). Using computers in the teaching of reading. New York, N.Y.: Teachers College Press
Tingey, B. & Thrall, A. (March/April, 2002). High stakes management. Multimedia Schools, 9(2), pg.S1-S7
Note: This article was written by the vice president of Successmaker.
Underwood, J., Cavendish, S., & Lawson, T. (2002) Integrated learning systems and educational outcomes. Available online at http://www.psychology.nottingham.ac.uk/research/credit/project/ils_...
L
Van Dusen, L. M. and Worthen, B. R. (1993). Factors that facilitate or impede implementation of Integrated Learning Systems. In G.D. Bailey (Ed.), Computer-based Integrated Learning Systems (p.115-119). New Jersey: Educational Technology publications, Inc.
White, M. (1993). Are ILSs good education? In G.D. Bailey (Ed.), Computer-based Integrated Learning Systems (p.115-119). New Jersey: Educational Technology publications, Inc.
Wiberg, E. A. (March, 1992). How to choose the right ILS. Electronic Learning, 11 (6), pg.10; New York.

• ▶ Reply

Permalink Reply by Joe Sassaman on March 9, 2009 at 12:00pm

I can see that you feel ILS (and SuccessMaker) are not as worthy or legitimate as Pearson (the mfg.) would have us believe. My problem with your data above is that many studies are very old. How legitimate is a study on technology that was done over 23 years ago (1986)? I know that there are more current studies cited but again you are citing very old outdated studies. Are you assuming that ILS and SuccessMaker haven't changed in the last 20 years? I appreciate all the info, but have you personally used SuccessMaker in an Elementary classroom in the last few years and if so what are your feelings about it?

• ▶ Reply

Permalink Reply by Mari Abes on March 18, 2009 at 7:07am

Hi Joe!

I work for Pearson and am the Account Executive for part of Florida. I definitely believe in making informed decisions so I will let you determined what is best for your school and students. I do work with Sarasota County Schools and they have expanded the SuccessMaker program to every school in the district. Please let me know if you would like contact information to talk to someone that is actually using this product.

Best of luck!

• ▶ Reply

Permalink Reply by Antonio Villar on March 25, 2009 at 6:16pm

Love the program. We are a K-8 grade school in San Diego, Ca. This is our first year using SuccessMaker, and we have had outstanding results so far. It's a supplemental program, one that we use for reading and math intervention, and not meant to take the place of the classroom teacher. I can tell you more if you wish, but let me just say that the students are very motivated by the program and have improved in their reading and math skills. We will have a final assessment of the program at the end of the year, but if current data is any indication, we have a key part of our RtI support in place.

• ▶ Reply

Permalink Reply by Ian Grove-Stephensen on July 7, 2010 at 6:35am

Joe, you started this thread over a year ago. I would be really interested to hear what you decided, and how it has panned out so far?

• ▶ Reply

Permalink Reply by Joe Sassaman on July 7, 2010 at 9:55pm

We did not purchase this program. After doing an on-line sales pitch with one of their reps, all I can say is that I was very underwhelmed. The graphics were very dated and lo-res. Kids, in my opinion, are very tech savvy these days and know what a good video game or web site looks like and this program needs a serious overhaul to stay current...

...that and what money we thought we had evaporated somewhere at our state capital...

• ▶ Reply

Permalink Reply by Ian Grove-Stephensen on July 8, 2010 at 12:09am

Thanks for the update!

• ▶ Reply

Permalink Reply by Danny G. Kelly on August 30, 2010 at 9:26am

I'll preface this by saying I'm a SuccessMaker rep for Pearson Digital Learning. If you've not looked at SuccessMaker recently, I would suggest giving it another look at www.successmaker.com. We've made significant improvements over the past 2 years. (many were way over due) We've also had some external companies perform some efficacy studies on the new math content and will be running a reading efficacy study this school year.

• ▶ Reply