GPTZero: Part of the Problem?

I have seen multiple discussions about running work through GPTZero in order to test for AI content and I can honestly say to you that this on the surface is a great idea, but please don't be naive when you think that it will always give you a correct result. Also don't be naive to think that AI hasn't developed past GPTZero's capabilities...

Let's clear the air, shall we?

Part 1: The GPTZero Problem

GPTZero wants you to pay for GPTZero and the more you use it, the more you realise that the free version is actually pretty bad compared to the paid version (obviously: but even the paid version is crap). And that there are better AI detectors out there. When you pay for GPTZero, you are paying AI because now that these AI scammers know you're using GPTZero, they are likely not smart enough to know most of GPTZero is false negatives/positives and thus, start paying for better versions of AI/using timed free subscriptions to it.

Let's give you a bit of a worked example, shall we? It doesn't take a long time to search the internet for people who have run their own work through GPTZero and it has come back as being written by AI. I have done this with some of my own work where I was quite astounded to learn that I am in fact, a robot. (I am not a robot, I just have issues). Everyone thinks they are a detective or a revolutionary, getting very emotional when they see the little circle light up yellow on the GPTZero website but here are some reasons why perhaps, you need a better (and possibly need to pay for a better) AI detector...

1) GPTZero often flags academic writing as written by AI because of the lack of expressive quality

Academic writing is often known for its bland expressions. Some of the authors will prevent themselves from having an objective opinion and would rather state a fact instead. This causes GPTZero to flag the content as AI even though it has in fact, been well-researched and written by the person themselves. It even flags secondary readings as AI generated content, which is odd and of course, incorrect. An in-text citation can't be concluded as AI. But GPTZero seems to think that it is. Truth be told, the more structured and academic a piece of writing looks, the more likely it is to be flagged as "AI generated" content.

2) The way it works is not really well explained, leaving people in the dark about how and whether it works

One look into the stratosphere of the internet and you will no doubt find the en masse hysteria against AI generated content - often by misdirected educators who don't understand the problem. But you will also find that many people do not know how to detect it from eye, and this is the most important thing. Of course, the unattributed AI content creator knows you're using a shoddy service called GPTZero and, due to its lack of transparency, aren't worried about getting flagged. If they are flagged, they can simply claim that GPTZero isn't accurate and then we are all back to square one.

But the honest truth is that a claim made with evidence from a service like GPTZero is not going to hold up when flagging something as AI-generated. Therefore, the soul of a man under unattributed AI is just going to try and get better at hiding it by making the occasional tweak so you will get off his machiavellian back. All we are doing therefore is telling on ourselves about the fact we know this is happening. They become better at hiding it and we get left to do more chasing - but this time the case is far more difficult to prove as adaptation is taking place.

3) Rewording, context and plagiarism are all part and parcel to showing us it doesn't work

I have analysed, like the rest of you, that this site is overwhelmed with unattributed AI content. Running these through GPTZero once upon a time, would've done quite the trick. I mean back then, AI was still a baby and didn't know how to walk yet. But now, with constant rewordings, even my list of 'words that AI tends to use often' has become obselete (I only made it about six months ago). On top of this, GPTZero can't detect context, again making the blandest essays from forty years' ago vulnerable to an AI tag - which is ridiculous.

So what do we do? Well, we learn about our fellow creators who write their own stuff, we create our circles of friends who want the same things we do and we monitor the changes in AI content. It's now no longer about writing, it's about rewording and shifting, it's about plagiarising and create salads out of other people's hard work.

My Ironic Story Which Made Me Laugh Until I Cried:

I have come across two such salads of my own 'why it's a masterpiece' series (which contains one of my works ironically, I put through GPTZero and now, I am apparently a robot 🤖) on this site, had them reported and they were taken down. But it was quite astonishing to read what had happened. This person had put the stories through ChatGPT and spat them out again without plagiarism upon them, and most ironically of all, GPTZero couldn't pick it up (but picked up mine that was written by me). I mean if that doesn't tell you it doesn't work then I don't know what will.

That's kind of where I lost faith in GPTZero.

***

The fact that structured, academic and clear writing that doesn't have human spontaneity is flagged as AI generated is about to make the academic world have a headache and a half. I can't blame them for being scared.

4) Probabilities over realities and the way in which pieces are flagged

GPTZero has this very original problem of guesswork being its primary go-to. Most people who use GPTZero with half a brain can understand that it is more often wrong than it is right. This is because AI is developing faster than detectors can catch it and because of the fact that GPTZero has learnt how to detect AI content based on low-quality human writing (social media etc). This writing is often very emotional and just displaying emotion or objective opinion can get a piece passed as human even if it is AI-generated. This is a whole new problem. What happens when we know a piece is AI generated, but GPTZero says it isn't?

Well, the best thing to do is go off your hunch. Even if you can't prove it, report it anyway so that Vocal (who I am sure have better tools than the average person) can actually have a look at it. The truth is, we are never going to be 100% accurate at detecting AI, but it is better to be safe than sorry. I have recently had to learn that the hard way and unfortunately have lost someone I deemed to be a good online friend to the world of AI-generated content that was not only unattributed but totally flew under my radar. I could've cried.

What I'm trying to say here is: your eyes are just as good as GPTZero, if not better. As I was saying, I once made a list of words that AI could pick up as AI-generated content and then, I would look for that word pattern. But with the speed of AI development, my list became useless very quickly. If you do a bit of reading around the internet world, you should come across some good tools to see how you can help detect possible AI content. The mass rush to use GPTZero is misdirected and again, we are just telling them to adapt more. It's a whole new level of wrong.

5) It's relied on by people who don't think that AI is changing as quickly as it is, therefore they don't understand how AI works and that's a whole new thing

Before I came into these circles of checking for unattributed AI I did some research. I love becoming immersed in random topics (and if you remember my deep-dive into consumerism where I read like fifty books on the subject, you'll know it can often become dangerous for my mental health). I read quite a bit about AI and how it works. The more we communicate with it, the quicker it becomes at mimicking human speech. More often than not, people who are writing unattributed AI content either know about this, or are helping to train the machine themselves.

People who do not understand that with every single passing day, the monster changes shape are more likely to become overly emotional revolutionaries instead of listening to the sound of the drum as the clock ticks. They are more likely to indirectly tell the generators that they have to up their game because people are watching. I recently published a review of my first 12 hours as admin of the VSS Facebook Group and even I regret that I told some key things within that would help the unattributed AI users. (I did however say that I wouldn't say everything and I think I've kept enough to myself for them to not have the entire story).

But, if we can keep this stuff to ourselves at most, stop acting like emotional student revolutionaries ready to be blown apart in the June Rebellion and concentrate on learning about the subject itself rather than having emotional reactions that can have quite horrifying consequences to both the trail we are on and the people we are, we can prevent ourselves from falling down a dark hole. We can also to more learning on the topic, thus making ourselves better at detection.

Part 2: What Have We Learnt?

• GPTZero is more often wrong than it is right, unless the person is really bad at using AI
• GPTZero is unreliable if the person has written something in which they reserve opinion and judgement but give a factual, academic look at things. (If you don't believe me, there are so many examples of people running their own academic work through GPTZero and learning that they are robots. Often, this work was written before ChatGPT existed, sometimes by decades. It's still fun to do though because it's hilarious).

Side story:

I actually even ran my M.A thesis through GPTZero and it told me it was 90% written by AI, despite being written almost a decade before AI was even about which it definitely gave me a good chuckle. So, it's a nice toy - besides that it's really nothing of value.

• GPTZero is the go-to for people who don't understand how fast AI has changed in the past year
• Publishing that you're using GPTZero is just going to make these people better at hiding themselves and imitating people through getting the machine to learn more writing.

Conclusion

The human mind is such a great thing in comparison to AI, so why are we using less capable AI to combat more capable AI? Why are we bringing a knife to a gun fight?

Read the books, learn the methods, strike the content.

Your eyes and your mind are your best weapons.

***

Post-Script Edit:

Since this has become a top story, please enter my unofficial poetry challenge!