Telltale Signs for AI-Written
|
|
|
Summary and Introduction
This webpage summarizes a multi-year experiment to identify the telltale signs of technical summaries written with artificial intelligence (AI). In this work, we found that the text for summaries generated with ChatGPT has a distinct style. As a limitation, this AI style is not necessarily the same as in other AI-written documents. Our work went deeper than commonly cited signs such as use of em-dashes and correlative conjunctions ("not only...but also"), which are simply writing tools not often taught in schools. For instance, our signs included beginning sentences with an unusually percentage of subject-noun phrases, avoiding the first person, and ending an unusually high percentage of sentences with participial phrases. Recognizing these signs is a valuable skill, which can among other things help you write better prompts. For instance, in our sampling, ChatGPT preferred not to repeat nouns in a summary. For that reason, rather than repeating the term "customer need," ChatGPT will sometimes substitute another noun (such as "priority") as a synonym. Because "customer need" has a specific meaning for technical readers, such a substitution is imprecise. To overcome this weakness, authors should instruct ChatGPT in their prompts not to insert synonyms for technical terms. Experiment to Identify Signs of AI Writing Since Spring 2023, my teaching team assigned our engineering design students to use the AI tool of their choice to write a summary. This summary is for an initial design report that the students have written and received feedback on. Overwhelmingly, as their AI tool, the students have chosen ChatGPT. Presented here are the stylistic and grammatical signatures [1, 2] that have arisen from studying summaries that the students wrote with ChatGPT. Although our work from 2023-2025 focused on ChatGPT 4, we are updating this page to address changes in ChatGPT 5. Shown in Figure 1 is a report summary written by an instructor in a traditional way. The main text of the report itself was written by a team of students using traditional means. Shown in Figure 2 is an AI version of the summary using the same report. In the AI version, the prompt was simple: "Write a summary of no more than 150 words for the attached report." The colors in the AI version indicate telltale signs as defined in the next two sections: stylistic signs and grammatical signs. |
Stylistics Signs of AI Writing
Grammatical Signs
Other Listings of Telltale Signs Several excellent articles present telltale signs of AI-generated text [3-6]. Most of these articles focus on general writing, such as in essays and emails. In general, overlap occurs between these lists and our lists. However, our focus on technical writing and in particular a summary of an engineering design report has narrowed the types of signs that we uncovered. In addition to our analysis of AI summaries developed by students, we have surveyed the students themselves on the telltale signs that they notice. These surveys have revealed two telltale signs of AI writing that match the findings of our experiment: imprecision (hallucination and exaggeration) and long words such as -ability nouns. Students also commonly cited three other signs: fluff, em-dashes, and correlative conjunctions (not only...but also...). Please note that the citing of these last two signs might have arisen because many of the surveyed students had not been taught these tools of writing in their previous writing classes. For instance, many students do not know that the em-dash receives its name because it is to the length of the letter "m" or that this piece of punctuation is simply a horizontal version of a parenthesis. |
Figure 1. Traditional summary of a design report. The text of the report, which was one of the strongest in the course, was written by a team of three students, and the summary was written by an instructor of the design course.
|
Figure 2. AI summary of the design report used for the traditional summary of Figure 1. This summary is a composite of many summaries generated with ChatGPT 4. Colors denote the telltale signs of the AI writing.
|
|
Analysis
Opening sentence with subject-noun phrase: This report... The team... Proposed concepts,,, Discovery Space staff feedback... The next steps. All five sentences begin with a subject-noun phrase. Beginning each sentence the same way leads to tiresome rhythms. Active voice verbs: ...details...identified...include...favored... involve. In all five sentences, the main verb is active. Although using active voice is generally a strength in writing, this unusually high use of active voice is a sign of AI writing. Imprecision: ...the creation of a children's science exhibit. The choice of the word "creation" is imprecise. Rather than "creation of," more precise wording would have been "design concepts for ...." The reason for this change is that the report presented only design sketches of exhibits and not the actual exhibit. |
Needless Complexity: ...PA...maintainability. In these examples, the choices of an abbreviation and long noun insert needless complexity into the writing. To overcome this weak stylistic choice, authors will need to adjust their prompts.
Ending sentences with participial phrases: ...aiming... prioritizing...ensuring...meeting: Three of the five sentences end with a participle phrase. Moreover, the last sentence ends with two stacked participle phrases. Although such a high percentage of sentences ending this way is not a weakness, it is a distinct sign. Imprecision: The next steps involve developing prototypes. The phrase "developing prototypes" is imprecise because in the design process of this report, the next steps were to generate refined concepts and then select one for prototyping. Prototyping then was the third phase (not the second phase) of the project. |
References
- Michael Alley, The Craft of Scientific Writing, 4th ed. (New York: Springer, 2018).
- Michael Alley, "Section 1: Grammar," Writing as an Engineer or Scientist (1997).
- Charlie Fink, "The Seven Deadly Tells of AI Writing," Forbes (25 June 2025).
- Smriti Mallapaty, "Signs of AI-generated text found in 14% of biomedical abstracts last year," Nature (2 July 2025).
- Sam Kriss, "Why Does A.I. Write Like...That?" The New York Times Magazine (3 December 2025).
- Callum Borchers, "Why AI Workers Won't Let Bots Do the Most Basic Tasks," The Wall Street Journal (25 November 2025).
* Other than the AI example in Figure 2, AI was not used to draft or revise this webpage.