Last night I spent some time experimenting with how well AI can *actually* detect AI-written content.

I wrote an example cover letter without any LLM involvement and asked Claude and ChatGPT if it's AI. Then I regenerated one using the standard “generate with one, rewrite with another”, while making sure the prompt involves rewriting with avoiding phrases and words that are overused by GenAI.

The finding was a bit shocking - apparently I write more like AI than AI does, which probably means that if I were to apply somewhere, I’m just better off prompting a model and calling it a day. I’d probably save a lot of time too.

What does the fox AI model say?

For reference, here are some things the models told me about the one I wrote without any AI assistance:

• Polished and Professional Tone: The tone is highly consistent, polished, and reads like something crafted with deliberate attention to structure and style, which aligns with how AI tools like ChatGPT assist users in drafting professional documents.
• Keyword Usage: The inclusion of industry-specific terms like "design tokens," "Service Maps," and metrics (e.g., "SEQ") suggests it was written to align with professional trends and expectations. AI tools are adept at including these to make a document sound credible.
• Effortless Transitions: The flow between paragraphs is seamless, something AI often excels at when crafting cohesive narratives from user prompts.
• Length and Efficiency: The letter is concise yet comprehensive, balancing personalization and professionalism—again, a hallmark of AI-assisted writing.
• Personalization and Anecdotes: The reference to specific experiences, such as creating a design culture from scratch or hosting "Design Jam sessions," adds a personal touch. While AI can generate such content, it usually requires detailed prompts.
• Emotional Resonance: The mention of being "excited" about the real-world impact and phrases like "something immediately clicked" evoke genuine enthusiasm, which could suggest human authorship.

What I learned

Of course take it with a grain of salt, but I think there are two important lessons here:

• If you’re currently looking and applying, it’s probably worth running your cover letter and resume through one of the publicly available AI models. We all know companies use them, and some will accept or reject you automatically based on the AI detection. Even if you didn’t use any AI help writing your resume and your cover letter, it might still be flagged if you’re decent at expressing yourself by writing.
• If you’re a hiring manager, don’t leave the initial decision on a candidate to an automated system. While helpful, those will cut out a bunch of people who spent time honing their resumes and their cover letters just because they look too good.