I keep noticing where the risk section sits in industry writing about AI in marketing. It comes after monetisation and before scaling, wedged in like a paragraph that had to be there. Manipulation risk gets a paragraph. Psychological effects on users and ethical concerns get one each. Relative to the rest of the text, this is exactly the space a known bug gets in a set of release notes: acknowledged, logged, scheduled to be addressed.
The risk is real. Meta has faced lawsuits over manipulation allegations.
What actually happens in these texts follows a template. The risk gets one sentence: manipulation risk in users with low self-esteem and unrealistic expectations, full stop. Then the texts offer fixes: transparency, regulation, education, better optimisation, as though adopting any one of them settles the matter.
I recognise the template because I have worked inside it. In my work I have seen often enough how a risk assessment runs: someone writes it down, rates it, assigns a measure to it, ticks a box and everyone moves on satisfied. A documented risk counts as managed. Whether it is actually solved is a separate question, and that gap tends to satisfy everyone, even critics, at least at the political level.
That process might do the job for a piece of software with downtime or a security hole, but it falls short for a platform that shapes how young people see themselves. The people who build these systems, and the people who reason about them this way, apply the wrong yardstick. They come from a world where every problem has a solution, and a problem without one yet gets one in the next release. A checklist cannot supply the judgement ethics calls for, and a transparency feature does not change what a product actually does to a user. Regulation, in this view, would only slow the pace of development and the success it is meant to protect.
What this leaves out is a plainer possibility. For some problems, better technology is no fix. What ends them, if anything does, is a decision to do something differently, more slowly or not at all. That option does not sound business-friendly, and the companies that build these systems do not think in those terms.
Years ago I talked with an engineer about a product that was technically possible but questionable. His reasoning has stayed with me since: “if we don’t build it, someone else will. So we better build it, because we can build it better than the others.” It sounded pragmatic at the time. On reflection, it avoided responsibility. He called that realism.
The firms that build these systems run on the same argument, scaled up. The risks are real, the technology is coming regardless, so building it yourself starts to look like the responsible choice next to standing aside while competitors do it anyway. That reasoning makes one assumption: that a technology whose core function is influence over a person’s decisions can be designed so that the influence it exerts produces only good outcomes.
I don’t know if that can work. These risks appear right there in the texts that also carry the opportunities; what differs is the weighting. They give the opportunities most of the space, leave the risks a subpoint, and treat the moral question as one more managed item.