Improvement of AI Writing
|
|
In November 2022, OpenAI released ChatGPT (version 3.5) to the public. Since the spring 2023 semester, our teaching team in a large course on engineering writing has monitored the improvement of writing that uses artificial intelligence (AI) through one simple assignment [1]. In this assignment, students have been asked to use AI to create a summary of a report that they have submitted and received feedback on. That feedback was for both the content and writing.
In the six semesters of this monitoring, the quality of the summaries written by AI was distinctively below the quality of the summaries written by most of my students. In these AI summaries, the biggest shortcoming was that AI did not recognize who the primary audience was: a technical manager interested in both the design and the design process. Given that flaw, the AI summaries failed because they did not emphasize the report's most important details. In the AI summaries, other weaknesses also occurred, such as technical imprecision (using priority as synonym for customer need) and highfalutin language (for instance, replaceability). Had the students used more detailed prompts and iterations with AI, they could have overcome those two weaknesses [2]. Moreover, with enough detail in the prompts and iterations, the students could have also had AI emphasize the most important details. However, the time needed to write those prompts and do those iterations would have required as much as, if not more than, the time to write the summary with traditional methods. With the release of ChatGPT 5.0 on 7 August 2025, the needed time to achieve excellence has changed. Now, even with a simple prompt such as shown in the image, one can achieve a summary that emphasizes the most important details. Moreover, with only a few iterations on smaller stylistic choices, one can achieve a summary that matches even the best summaries written in the class. What has occurred is that this new version of ChatGPT recognized from reading the report not only who the intended audience was but also what details in the report most interested that audience. Moreover, the summary written by ChatGPT 5.0 could still be understood by the secondary audiences (who were less technical). After the release of ChatGPT in November 2022, it was clear that artificial intelligence would be able to communicate engineering and science to less technical audiences who were not as focused on the precision or emphasis of specific details [3]. Now, the release of ChatGPT 5.0 indicates that artificial intelligence is claiming yet a significantly larger slice of the writing that engineers and scientists do. How long will it be before AI (with thoughtful prompts) is writing effective abstracts and literature reviews for research articles? |
Improvement of AI as perceived by our team of readers
Common example of imprecision in the AI summaries.
Simple prompt that we used to arrive at respectable summary.
|
References
- Michael Alley and Samantha Splendido, "Three-Year Analysis of AI Report Summaries Written in an Engineering Design Course," submitted to 2026 ASEE National Conference.
- Michael Alley, The Craft of Scientific Writing, 4th ed. (New York: Springer, 2018).
- Catherine Berdanier and Michael Alley, "We still need to teach engineers to write in the era of ChatGPT," Journal of Engineering Education, vol 112, issue 3 (July 2023), pp. 583-586.
*AI was not used to draft or revise this webpage.