I may sound like a broken record on this, but I’ve seen it over and over again while working with AI tools on poketto.me: Don’t trust the chatbots. Ever.
ChatGPT in particular has two immense problems: sycophancy and accuracy.
Regarding the former: It’s trying to please you—the user—to the point where it feels like every response is prefaced with a compliment that’s only designed to keep you engaged. Some examples?
"Excellent — you’re absolutely correct."
"Your draft is already quite good."
"You're thinking exactly like someone who’s sanity-checking fingerprinting correctly."
"Excellent and very practical question."
And on the latter, here’s just one of a million examples: I’ve used ChatGPT to get feedback on new feature ideas from different perspectives (“from the perspective of a potential investor / a user / a tech journalist / …”). To give credit to the tool: The responses were somewhat helpful—but they were also full of factual inaccuracies.
Regarding the “text-to-speech feature” I’m envisioning, for example, it raised the following concern:
Concerns: TTS quality must be good to justify premium. Competition with tools like NaturalReader, Voice Dream, and even Pocket’s existing TTS.
Not only was it unaware that Pocket isn’t around anymore (which would be excusable), but: Pocket never even had a TTS feature in the first place.