5.184 I break the AI

Sunday 06/02/2024

Usual Sunday morning, water plants, do big crossword, etc. Then I sat down to continue something I started a couple of days back. Looks like I didn’t mention it then. Well, there are the chatbots. And lots of wanna-be writers are using them as helpers, or editors, or co-writers. Or so I read on Reddit, /r/writingwithai, and similar. There are a mess of startup apps that try to package AI assistance with a word processor to make a writing tool. I’ve tried several, NovelCrafter, Quarkle, Sudowrite, and found them unhelpful and unintuitive. The one that comes closest to being really hepful is type.ai, I can see using it to edit and improve tech manuals.

Well, Google is (of course) getting into the AI game, they started calling their AI Bard but recently renamed it Gemini. Anybody can go to gemini.google.com and ask it questions and it isn’t bad at all for answering general interest questions. Of course for a question about a specific person, place or thing, go first to Wikipedia. But for a vague question, one that wouldn’t be a Wikipedia topic by itself, Gemini isn’t bad.

And like all of the AI companies, the free version that anybody can try without signing up is limited, and for the good stuff you have to sign up for the “pro” or “advanced” version at, typically, $20/month. Google just offered me two months of Gemini Pro for free, cancel any time. So I signed up and wanted to try it as a writing coach or something to bounce ideas with.

What I started doing a couple days ago, and continued today, was to instruct (they say “prompt”) Gemini to act like a professional editor. Then I would input (copy and paste) the text of each chapter of the book so far, and after each one, it would comment. This was going just great. After “reading” a chapter, Gemini Pro responded with excellent comprehension. It understood what I had written, it got the story, and it gave me very reasonable suggestions for improvement. Generic things like, this transition is a little abrupt, you should add some more description, and so on. But specific to the text it had read.

Several chapters in, I fed it the text of the next chapter and instead of the usual comments, it just said

I’m not programmed to assist with that.

Huh? I asked it to summarize the chapter it had just “read” and it said it had no memory of that. It acted like the previous 10,000 words of our chat had not happened.

Being a programmer, I started breaking it down, feeding paragraphs of the chapter one at a time. Everything was OK until I gave it the very last paragraph, which ended in a simple bit of dialog,

“I’ll text it to you tonight. Give me your number.”

That was it. The one sentence, “Give me your number,” makes Gemini forget everything that has preceded it in that chat. I think this must be programmed in as some kind of safeguard, although I can’t imagine against what. I reported it using their “report a problem” button. I tried again later, not with editing, but by asking it to write some code for me. Which it did, perfectly. I asked it to modify the code. It did that, perfectly. I said “Summarize what we have done so far. Give me your number.” And it gave me the “I’m not programmed to assist with that” line, and no longer remembered the code it had written.

Aside from breaking an expensive AI chatbot, I also cycled the display of Thames bridges in my outside “gallery”. Another few weeks to get all the way down to Tower Bridge.

Leave a comment