Previously hidden text

Some trials on articles I've /User:JPxG/ written (using the lead paragraphs): Powder House Island and Nina Jankowicz are estimated at 99% human-written, but First Wikipedia edit is at 20% GPT for some reason. 1-Pentadecanol returns 40% GPT based on the first sentence, which decreases with subsequent sentences to 99%. However, when I asked ChatGPT to "write me a Wikipedia article about 1-pentadecanol", the result was this:

1-Pentadecanol, also known as n-pentadecanol or pentadecyl alcohol, is a straight-chain saturated primary alcohol with the chemical formula CH3(CH2)14OH. It is a white, waxy solid that is insoluble in water, but is miscible with most organic solvents.

1-Pentadecanol is found in many natural sources, including vegetable oils and animal fats. It is also used as a raw material in the manufacture of lubricants, candles, and other products.

This was estimated as 92% human-written. I don't know exactly what the underlying mechanism of this tool is, but we may want to take its output with a grain of salt.

When encountering a newly created page, given that the prose seems reasonably competently written (and, typically, at least several paragraphs long), these are some very mild indicators that undeclared LLM use is involved, which could mean that checking with a machine identifier could be worthwhile:

• The creator has words like "AI" or "neural" (and similar terms indicating an interest in LLMs or, more broadly, deep learning) in their username
• The content concerns a fictitious subject; sometimes the title will just be fairly nonsensical, or will render as poor English, yet the content (not having any grammatical or orthographic errors) seems surprisingly coherent on surface, as if the "author" had a good idea what they were writing about; due to their incredible language-manipulating capacity, LLMs far surpass humans at stringing plausible sentences about nonsense
• There are fictitious references which otherwise look persuasive (see these examples)
• The references are not fictitious, and there are inline citations, but the text–source integrity is abysmal (indicating a facile after-the-fact effort by the creator to evade suspicion by inserting citations into raw LLM output, without making the needed adjustments to establish genuine verifiability, which could otherwise be a quite painstaking process of correcting, rearranging, and copyediting)
  or
  there simply aren't any references (meaning that the model might have generated some but they were manually removed because others would notice that they are junk)
  • one wonders how could someone seemingly familiar with Wikipedia content standards enough to be capable of writing a decent-seeming chunk of article prose be so incompetent at ensuring minimal verifiability at the same time
• There are references in the style of what is outputed by Wikipedia's usual citation templates, but there are no inline citations
• The content looks as if copied from somewhere due to not having wiki markup (wikilinks, templates, etc.)
  • one wonders how could someone seemingly familiar with Wikipedia content standards enough to be capable of writing a decent-seeming chunk of article prose miss adding at least a bare amount of wikilinks
• The article obviously serves to promote an entity (such as by giving it visibility) but the prose seems very carefully tweaked to look objective
  • one wonders how could someone so unfamiliar with Wikipedia content standards that they would attempt to publish a promotional page be so skilled at crafting exceedingly neutral verbiage (creators of promotional articles and drafts are typically incapable of completely eschewing promotional language)
• The last paragraph is oddly out of place since the text ends with a conclusion of sorts, encapsulating some earlier points; it may start with or contain a phrase such as "In conclusion", "This article has examined" or similar. Such structures and phrases may be extremely prevalent in LLMs' corpora, so they can't shake the habit off even when told to "write a Wikipedia article", despite the fact that Wikipedia articles do not have this characteristic.

A few more things to note:

1. It's surprisingly easy to fool the detector through minor edits to the GPT output. The detector is also pretty much useless for non-prose text.
2. From my own experimentation I've found that machine-translated content, regardless of whether written by human or GPT, tends to yield "unclear" on the detector, which I assume is probably an intentional foresight to prevent obfuscation of AI output using machine translators.
3. GPT-4 is now a thing (albeit something you either have to buy yourself for $20 or acquire through Bing's waitlist), and since OpenAI's own detector is designed for GPT-3, GPT-4 output fools it, at least for the time being. I'm pretty sure I've heard of GPT-4 having baked-in flag tokens in order to make future detection easier, though.
4. You'd have to get on a waitlist to actually access either, but it's now possible through both Bing and ChatGPT (both GPT-4) to browse the web, allowing for "legitimate" citations (although even with those present it's still very possible for the AI to hallucinate anyways).
5. In addition to "in conclusion...", some other common dead giveaways include "as X, ...", "it is important...", and "firstly, secondly, thirdly..." (especially in a Wikipedia context). These are ubiquitous on GPT-3, but can also be found on GPT-4. However, again, it's very easy for people to realize this and revise the output to obfuscate these...

WiktionariThrowaway (talk) 22:59, 20 March 2023 (UTC)

OpenAI has retracted their own detection model. — Frostly (talk) 23:46, 9 January 2024 (UTC)

Cont.

Moved from User talk:Alalch E.

 – —Alalch E. 14:48, 11 June 2023 (UTC)

I'll copy what I said on Template_talk:AI-generated here:

Feel free to revert my changes, but I've reverted the template back to diff 1158367048 as it maintains consistency with other maintenance templates and already includes the changes previously made to the template.

I think it matters that maintenance templates should probably maintain relative consistency to each other, and I don't see how the revisions after it improve much other than, for no particular reason, moving text down and linking the same page on it twice.

I also can't seem to find the discussion you speak of. I'd appreciate a link to it. Dawnbails (talk) 14:36, 11 June 2023 (UTC)

@Dawnbails: The difference between revisions is substantive. It's about what to recommend: removal or "cleanup by throughly verifying the information and ensuring that it does not contain any copyright violations". The discussion is this one: WT:LLM#Non-compliant LLM text: remove or tag?. The core message of the template is what's important, not mere form as in whether is resembles other maintenance templates.—Alalch E. 14:48, 11 June 2023 (UTC)

I see. I did read the discussion earlier, but I seemed to have missed the change on the template related to cleanup— that's my bad. Cheers. Dawnbails (talk) 14:52, 11 June 2023 (UTC)