Today's AI models are trained to turn down many requests, from dangerous instructions to sensitive topics. MIT Technology Review argues that this habit of refusing is not enough as a safety mechanism. Refusals are often bypassed with clever prompts and do not address the deeper risks. Real safety needs several layers, not just a chatbot saying no.
What it means: The MIT Technology Review argument touches something we see in almost every project built on language models: teams rely on the model refusing harmful requests and skip the ordinary controls they would put on any other system, such as per-user permissions, action limits and audit logs. A refusal is a behaviour the model learned from training data rather than a rule written in code, so it can shift with a new model version, a reworded request or instructions hidden inside a document or web page the model reads. For a business putting a chatbot or agent in front of customers, the practical lesson is to give the model access only to what it needs, route sensitive actions such as refunds, account changes or outgoing emails through a separate check or a human approval, and log every tool call so it can be reviewed later. That way, even if someone does manage to trick the model, the damage stays contained, and safety no longer depends on whether a statistical system happens to say no at the right moment.
Read the original on MIT Technology Review
Related service: Custom software