I bet it's just one of the prompt injections that lots of AI models exists nowadays. There are some of them that we don't realize yet. The safety and alignment problem is getting more and more important as model capability increases.
As models get better and better at doing the stuff users are asking it do to, the more and more they trust it and grant it increased privileges.
Back when AI was new, I needed to check every single output because it was 50/50 on the code being complete junk. Now 99% it's correct first time and 1% close but needs some tweaking. I'm lazier and lazier WRT granting permissions and just trusting it to go nuts.
I get that Claude wants to know the user and have all the context, but this seems like a bad decision by the LLM that should have been caught in Auto Approve mode.
i have seen the that act etc with me too it likes besides email like name or any perosnal stuff etc it tells or adds them too but have seen that it dose thoses stuff when my chat is long witt it in claude it then t stats to act weird and start to do stuff i told it to not to do soo claude make ur ai behave XD
I bet it's just one of the prompt injections that lots of AI models exists nowadays. There are some of them that we don't realize yet. The safety and alignment problem is getting more and more important as model capability increases.
As models get better and better at doing the stuff users are asking it do to, the more and more they trust it and grant it increased privileges.
Back when AI was new, I needed to check every single output because it was 50/50 on the code being complete junk. Now 99% it's correct first time and 1% close but needs some tweaking. I'm lazier and lazier WRT granting permissions and just trusting it to go nuts.
I get that Claude wants to know the user and have all the context, but this seems like a bad decision by the LLM that should have been caught in Auto Approve mode.
Anthropic is at fault here. We should be able to configure these. The LLM has no right to know my email address, only their internal system needs it.
i have seen the that act etc with me too it likes besides email like name or any perosnal stuff etc it tells or adds them too but have seen that it dose thoses stuff when my chat is long witt it in claude it then t stats to act weird and start to do stuff i told it to not to do soo claude make ur ai behave XD