This is what I hate about people trusting it. If you rely on AI to operate in a domain you don't man-handle, you will be tricked, and hackers will take advantage.
"AI! Write me gambling software with true randomness, but a 20% return on average over 1000 games"
Who will this hurt? The players, the hackers or the company.
When you write gambling software, you must know the house wins, and it is unhackable.
A better example would be to use LLMs to generate passwords or secret keys. Then even if it looks random to human, the inherent bias would make it a security disaster.
You just went and created the worst example. The model knows how to create an rng, that's not it weakness. In fact, if you give it a random mcp it won't do that.
If you use AI to write a gambling software you run in production without reviewing the code or without a solid testing strategy to verify preferred odds, then I have a bridge to sell you.
"AI! Write me gambling software with true randomness, but a 20% return on average over 1000 games"
Who will this hurt? The players, the hackers or the company.
When you write gambling software, you must know the house wins, and it is unhackable.