(Updated April 14, 2022: amendments made to summary of Ghufron & Rosyida's study)
Grammarly is a grammar-checking tool that is becoming increasingly popular among students in the ESL/EAP/English language learning field. A survey of my course this session found that 50% of students were using it. As an instructor who values academic integrity, this made me concerned about two issues:
- What effect does Grammarly have on language acquisition?
- Is using Grammarly academic dishonesty (cheating)?
The second issue is particularly of concern for online courses, where students write exams on their home computers.
Literature Review: Grammarly, Error Reduction, & Language Acquisition
A 2018 study by Ghufron & Rosyida in Indonesia focused on the effect Grammarly has on reducing student errors. Forty students were divided into two groups: the study group which used Grammarly throughout a semester, and the control group which received indirect corrective feedback from their teacher. (Note: Indirect corrective feedback means the teacher highlighted problem areas in the students writing, leaving the students responsible for correcting their errors.) At the end of the semester, the students submitted a final writing task after receiving feedback from either Grammarly or an instructor. The results were as follows:
- Students who used Grammarly produced significantly fewer errors in the writing task at the end of the semester compared to those who had indirect corrective feedback from their teacher. There were fewer errors related to vocabulary use, grammar, spelling, and punctuation.
- Grammarly is less effective in improving content and organization of student writing as compared to a teacher, as the software cannot detect the appropriacy of written content.
Overall, the study concludes that Grammarly can be helpful in reducing errors.
A 2019 study in Australia by O'Neill & Russell analyzed 1) how academic learning advisors viewed the feedback provided by Grammarly and 2) how Grammarly's feedback compared to feedback provided by an advisor. It found that advisors viewed the feedback provided by Grammarly as "much more useful" than that given by human advisors (p. A-98). They also noted Grammarly's time-saving value, as one key reason why advisors cannot/do not provide detailed grammar correction is due to the time it takes; however, by using Grammarly, grammar corrections can be generated almost instantly. This leaves advisors with more time to focus on global content issues (e.g. referencing, argument, coherence).
A 2018 study by Fadhilah, Lizawati, and Saribu in Indonesia studied the effectiveness of Grammarly in improving the quality of written abstracts. By comparing pre-test (without Grammarly) and post-test (with Grammarly) results, the researchers found that the quality of abstracts increased significantly with Grammarly as an aid. (Author's note: This may not indicate learning; it may only show that following the suggestions of the software can improve a text.)
A 2016 study by Qassemzadeh and Soleimani on Iranian EFL students compared the effect of feedback from Grammarly with feedback from an instructor. It found that there was not a statistically significant difference between the effectiveness of the two.
Summary of Literature
- Grammarly appears to reduce student errors. However, it is unclear whether this indicates actual learning is taking place.
- The adoption of Grammarly can save instructors time when generating student feedback.
Do Students Learn from Grammarly?
The above research shows Grammarly use reduces errors. However, it does not show that using the software for a prolonged period prevents students from committing errors. In other words, it does not indicate that students become better writers when unassisted by the software. This issue needs to be investigated.
How Effective is Grammarly's Feedback?
Before we discuss if using Grammarly is academic dishonesty, it's important to review the extent to which the software is able to correct mistakes. In a survey of my students, I have found that most students use the free version (i.e. not Premium), so that is the version we'll analyze here. Here is Grammarly's feedback on ten writing mistakes taken from this website's 10 Common Writing Mistakes post.
As we can see, the free version of Grammarly ignored the following errors:
- 1. comma splice (run-on sentence)
- 2. fragment
- 5a & 5b - faulty/poor parallelism
- 10. awkward word choice (according to me)
Below are Grammarly's explanations of the highlighted errors. As you can see, corrections are directly given to the student.
Other than its suggestion for footnote #7 (there is no error in determiner use), the corrections are accurate. The software is able to identify subject-verb-agreement errors and preposition and article usage problems.
On the other hand, it seems that sentence fragments and run-on sentences are not detected in the free version, which is unfortunate since these are common errors in college-level writing.
Note: The software did find eight additional mistakes that require the Premium version to see. The "Punctuation in Compound/Complex Sentences" and "Incomplete Sentences" issues likely cover the run-on sentence and fragment mistakes. The other suggestions may cover the other mistakes as well.
Lastly, though not shown above, the software does correct spelling and small punctuation errors (e.g. forgetting a period, too many spaces), which is also of value. Here's an example:
How does Grammarly Compare to Microsoft Word?
Here is the same list of ten mistakes in Microsoft Word:
As you can see, Microsoft Word (version 16.37 for Mac) only highlights "it's" and "cuz", which would give it a score of 1.5/10 compared to Grammarly's 6/10.
Summary
- Though the free version of Grammarly cannot detect some common writing errors, it was about to identify and suggest an appropriate correction for 6/10 of the common writing mistakes tested, which would likely be useful for a student.
- It is effective in reducing spelling and small punctuation issues.
Question #2: Is Using Grammarly Cheating?
To answer this question, consider this scenario presented in the EAP textbook Advanced in Academic Writing by Marshall (2017):
A student asked for help from a private tutor, who corrected only grammar and vocabulary errors so the student could get a better grade.
1 - plagiarism2 - not sure3 - not plagiarism
As for whether this is plagiarism, the answer key states:
Instructors have different policies about getting help from private tutors for error correction, so it is always a good idea to check. However, when students' academic language is assessed, such as in EAP courses, many instructors would consider this cheating.
In other words, for assignments that test vocabulary and grammar, it is dishonest to submit vocabulary and grammar produced by another person (or machine). I agree with Marshall, and I imagine most college instructors would.
This raises the key question: When Grammarly is enabled, is the language produced a result of the student or the program?
I don't think we can deny that the software itself, not the student, increases the quality of vocabulary and grammar. The software discovers errors and presents the solution. This is not an indirect method that requires the student solve the issue; the answer is directly given. In a strict sense, this would be academic dishonesty (cheating) because it is essentially the same as the private tutor example above. Student scores will be higher because of a 3rd party, not their own ability.
While this seems like the logical conclusion, it creates several contradictions and counter-arguments:
1. Spell-checkers (and possibly dictionaries before that) were once new technology that could be viewed in the same regard.
2. Grammar checking with Microsoft Word is already an accepted practice in student work.
3. Grammarly use may help students learn*
* This is a key issue that needs to be investigated further.
4. Banning Grammarly would be difficult. No watermark is left on the text so there is no definite way to know if a student has used it. The instructor would have to make accusations based on unexpected language improvements, which is challenging and often leads to a draining investigation.
5. Standards in language assessment need to evolve.
Instead of seeking to ban learning aids, academic institutions should adjust their evaluation methods to welcome technology and AI-based software. This could be done, for instance, by
- relying on paper-based in-class assessments to evaluate language skill (in times when there is not a global pandemic)
- putting less emphasis on language in assessments and more on content and other aspects
- recognizing that the future will be a world of enhanced technological assistance, and therefore the need for students to prove their true writing ability with a pen and paper will become rarer and rarer.
Conclusion
Regarding its educational value, studies show that Grammarly can benefit instructors by saving them time though it is unclear to what extent Grammarly enables students to learn compared to traditional approaches.
Regarding its effectiveness, the free version of Grammarly is able to identify considerably more grammar mistakes than Microsoft Word.
Regarding academic integrity, the argument can be made that using Grammarly, even the free version, could be considered cheating for courses where language is assessed (e.g. English or EAP classes). Despite this, considering the benefit the software provides students, the barriers to banning it, and the value of accepting new technology, educators should make steps towards welcoming computer-assisted learning tools such as Grammarly in the educational field. This would be useful for both students and instructors.
Key Takeaways
- For students: If grammar and vocabulary are being tested in a course you are taking, then check with your instructor first before using Grammarly on assignments or exams.
- For instructors: Consider revising course guidelines to allow for the use of computer-assisted learning tools such as Grammarly for take-home assignments.
Written by Matthew Barton, creator of Englishcurrent.com (copyright)
References
- Fadhilah, U., Lizawati, & Saribu H. J. D. (2018). Effectiveness of Grammarly Application for Writing English Abstract. International Journal of Science and Research (IJSR), 8(12), 163-166. https://pdfs.semanticscholar.org/d6c7/7e006927782e8dcb6d4571ea83cc0c9fdf90.pdf
- Ghufron, M. A., & Rosyida, F. (2018). The Role of Grammarly in Assessing English as a Foreign Language (EFL). Writing. Lingua Cultura, 12(4), 395-403. https://doi.org/10.21512/lc.v12i4.4582
- Marshall, S. (2017). ADVANCE in Academic Writing. Pearson.
- O'Neill, R. & Russell A. (2019). Grammarly: Help or hindrance? Academic Learning Advisors' perceptions of an online grammar checker. Journal of Academic Language & Learning, 13(1), A88-A107. https://journal.aall.org.au/index.php/jall/article/view/591/435435452
- Qassemzadeh, A. & Soleimani, H. (2016). The Impact of Feedback Provision by Grammarly Software and Teachers on Learning Passive Structures by Iranian EFL Learners. Theory and Practice in Language Studies, 6(9), 1884-1894. https://pdfs.semanticscholar.org/547f/a6a35f4cdc6387f0eba2347c72b52ed2e76d.pdf