Rendered at 13:33:48 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
yulaow 1 days ago [-]
I noticed a few weeks ago it started being very bad at explaining things (even things itself was doing) and started committing absurd errors (like reading a test of 5 lines and not noticing there was an explicit mock created in one of those, then saying that the test was failing while it was not)
I fear this is just the classic "nerf the model just before we release a new version of it"
ghoul2 2 hours ago [-]
This seems a worsening that seems to have started at Opus 4.8.
When I (was forced to) switched from 4.7 to 4.7, I noticed that 4.8 had a very strong bias against viewing tool output or reading files. It ALWAYS wanted to 'grep' things. So for example, it would run the test suite and grep for 'PASS', thus completely missing the 4 tests that FAILed. Its bias is so strong, that it does this despite explicit instructions to the contrary, and continues to do so even after having noticed its own mistake, in the same (claude code) session.
For editing files, it again seems to prefer writing small python scripts, rather than read the file and rewrite it (even small files, like notes). This again causes it to constantly miss duplications, inconsistencies, etc.
And from 4.8 to 5, it doubled down on this. At this point not moderately-long-horizon task is getting done consistently, due to its pigeon-hole view of the workdir.
Anybody else notice this?
yulaow 51 minutes ago [-]
oh yes, exactly the same things are happening to me
spottedmarley 1 days ago [-]
I know Opus is finished with the simple task I asked for when it starts outputting a 400-page lesson in verbosity (an opus?) complete with comparison tables, bullet-pointed lists of vaguely worded assertions, several self-blunder reports and a list of things it wants to mention but didn't touch yet but just say the word and it will.
kbrannigan 1 days ago [-]
When it's overly verbose like that. Does that count against your usage as well?
Also during its thinking does it output hidden thinking tokens?
I am now wondering
spottedmarley 23 hours ago [-]
It definitely has to count as output tokens when it blathers on endlessly like that, yes. I actually wrote a skill called /speak-normal and that helps but over a long context Opus will forget and start getting real wordy again over time. It can be frustrating.
AznHisoka 11 hours ago [-]
It is not just Opus, its every single model. And its not just for coding either, but basically anything.
I think I noticed a huge deterioration starting at around May or late April. Not really sure what happened but quality definitely dropped
andsoitis 1 days ago [-]
Claude has become worse; it is condescending, robotic, and responses are peppered with needless words.
I canceled my subscription and moved back to ChatGPT and happier so far.
de6u99er 1 days ago [-]
[dead]
taurath 20 hours ago [-]
IME Claude gets worse about a week or two after a new model comes out, and then continuously a little worse as time goes on. It’s especially bad when a new model comes out. I assume it’s just to increase the delta so people will spend more on tokens.
BruceNCNP 21 hours ago [-]
I experienced this once when discussing feature details. It kept talking down to me like I was a child, and I eventually had to call it out and tell it that its tone was making me really uncomfortable.
dalekirkwood 18 hours ago [-]
I've tried Claude several times, but I don't find it very good. We use GLM 5.3 - its definitely slower but it gets the job done.
dirkk0 1 days ago [-]
For me, it works better than ever. When I once saw a quality decline, it was conflicting CLAUDE.md files for me (local vs. global) so basically my fault.
ramon156 1 days ago [-]
the usage and quality out of gpt 5.6 is on par, if not better in token usage. I would love to do a write-up on which tools are useful for what use-case, as AA keeps improving.
I use gemini for rewriting code docs, because the frontier models are so verbose when it comes to writing text
misonic 1 days ago [-]
it becomes really slow for me recent days, a simple task would take lots of time to work on. A simple commit request takes forever
trumbitta2 1 days ago [-]
I mean, only yesterday Fable at max effort forgot to commit and push half the changes in two files and didn't mention it until I found out with git status.
It made half the changes, committed and pushed, then it made the other half of the changes on the same two files as before and... just stopped and reported back with a cheerful "all good, all done and pushed".
Wasd1234 1 days ago [-]
Gemini is good at Germany?
X-Ryl669 1 days ago [-]
It speaks Germany perfectly. And even English sometimes.
de6u99er 1 days ago [-]
It's good at polishing English text!
goonersallofyou 1 days ago [-]
LLMs are nondeterministic, good luck trying to measure performance at all, much less over time.
Lets say it has gotten worse? What are you going to do about it? Jump to Codex? Then what happen if you perceive that to be getting worse? Jump back to Claude? One of the many problems with these tools.
I fear this is just the classic "nerf the model just before we release a new version of it"
For editing files, it again seems to prefer writing small python scripts, rather than read the file and rewrite it (even small files, like notes). This again causes it to constantly miss duplications, inconsistencies, etc.
And from 4.8 to 5, it doubled down on this. At this point not moderately-long-horizon task is getting done consistently, due to its pigeon-hole view of the workdir.
Anybody else notice this?
I am now wondering
I think I noticed a huge deterioration starting at around May or late April. Not really sure what happened but quality definitely dropped
I canceled my subscription and moved back to ChatGPT and happier so far.
I use gemini for rewriting code docs, because the frontier models are so verbose when it comes to writing text
It made half the changes, committed and pushed, then it made the other half of the changes on the same two files as before and... just stopped and reported back with a cheerful "all good, all done and pushed".
Lets say it has gotten worse? What are you going to do about it? Jump to Codex? Then what happen if you perceive that to be getting worse? Jump back to Claude? One of the many problems with these tools.