Several of our high school English teachers have been asking about automated writing scoring programs to help them score student essays. I'd love to hear what others think about these kinds of programs or hear how they are being used in schools. I am just beginning my research on this topic, so any advice is appreciated.
It will not be that accurate and lenient to evaluate.
I think automated scoring would not be as accurate. One of the hardest part of grading writing is that it is subjective. Even if a rubric is used, two people grade differently. Although a program would be easier, it may not be necessarily give a score you would agree with. It would actually create more work because you may end up double checking the scores.
I think it's important to start with the question "what am I trying to assess?" Some programs are designed to spot and penalize poor grammar, syntax, spelling and other mechanics. Others know to look for transition words and likely thesis statements. Chances are, though, a HS English teacher is looking for much more than this. For example, evidence of critical thinking; effective and appropriate use of logos, ethos, and pathos in a persuasive argument; seamless use of source material. Auto-graders will struggle with that, though some claim they won't.
One way to see the limits and possibilities is to check out a description of Pearson's Writing Space auto-grading feature. They make bold claims about what it can do, but you can't write your own prompts and would need to choose from a bank. So, there are trade-offs.
Here is their description (link to page):
Auto-graded writing assignments use artificial intelligence to assess your students’ writing assignments. These assignments are specific to your course, require students to submit their work through Writing Space, and are automatically graded. Writing Space not only checks spelling and grammar, but evaluates content and quality using the grading rubric specific to the question. This may sound miraculous, but the artificial intelligence behind Writing Space is what enables Writing Space to assess content in your students’ essays: