I keep running into the same question in the AI debate: does this technology serve the customer, or does it serve our business interests? It is the right question. I have never once heard it answered honestly.

Put the question to a room full of marketers and the answer arrives within seconds. Personalisation. Customer experience. Individual outreach. Customer-centric action. The vocabulary shows up so fast that it smooths over the tension before anyone names it.

The tension is real, and it has been real the whole time. Every personalisation system a company runs is built on data the customer did not hand over knowingly, or handed over without knowing what would happen to it next. Personalisation means a company knows things about you that you never told it, and uses that knowledge to sell you more.

That mechanism sits in the marketing textbooks that train the people who build these systems, stated plainly enough once you look for it, just wearing a friendlier name in public. The public version is “we give the customer what they need before they know it themselves,” offered as service. It describes collecting and acting on data the customer never knowingly gave.

The engineering is plain. AI reads a customer’s data and turns the pattern it finds into a prediction: when this person is likely to buy, what they will need next, how to phrase the message so they stay. The words chosen for all of this are warm: relevant, individual, timely. The mechanism underneath runs on the same logic as any act of manipulation. I know something about you that you do not know I know, and I use it.

Ethics does come up in these conversations, arriving as a side note tacked onto a strategy already decided. A side note is optional: engage with it or skip past it, the strategy proceeds unchanged either way. A company builds the product plan first and attaches ethics afterwards, the way it attaches a certification sticker to a finished product.

I have worked in companies where the ethics discussion happened after the product decision had already been made. The discussion skipped over whether to do this and went straight to how to communicate that we were doing it. What ran under that sequence was public relations, dressed in the vocabulary of ethics.

The people running that sequence work inside a sales logic, and that logic resolves the tension between what serves the customer and what serves the business before the conversation even starts. The company asks the question and answers it too, and the question always resolves the same way: the business, every time.

Personalisation could serve the customer under different conditions. The customer would choose which data gets collected, see what the AI had inferred and challenge the pattern built from their behaviour. None of that would move the conversion rate upward, and the conversion rate is the number companies track, report and optimise against.