AI Regulation Tracker  /  Enforcement and data protection

Korea Orders an AI Model Destroyed After Data Was Misused to Train It

In January 2025, South Korea's data regulator did more than fine two companies. It ordered the deletion of an AI scoring model built on the personal data of 40 million people who never consented to it. This is the enforcement move executives keep saying could not happen. It happened.

By Anthony Guerriero  ·  The Leveraged Years Regulatory Research  ·  Published July 21, 2026  ·  Last updated July 21, 2026

Most executives I talk to treat AI training data the way they used to treat a marketing list. Something you acquire, load, and forget. South Korea just showed why that instinct is expensive. In January 2025, the Personal Information Protection Commission resolved a case that had been building for a year, and the outcome was not the usual slap. It was penalties of about KRW 8.375 billion across the parties and, more importantly, an order to delete the model that had been trained on data the regulator said should never have moved.

The underlying facts are simple enough to follow. Kakao Pay, a large Korean payments provider, transferred personal data on roughly 40 million users to Alipay. That transfer happened without the consent or disclosure Korean law requires for sending personal data overseas. Alipay then used that data to build a scoring model, reported as a non-sufficient-funds or NSF score, used in connection with Apple Pay. So the chain runs from an unlawful transfer, into a model, into a live product. The PIPC did not accept that the model could be quarantined from the violation that produced it.

What did the regulator actually order?

Two things worth separating. First, the money. The PIPC imposed administrative penalties totaling about KRW 8.375 billion, with the largest share falling on Kakao Pay for the transfer itself and a further penalty on Apple tied to disclosure failures. Fines like this are common and, frankly, priced in by large companies as a cost of doing business. Second, and this is the one to circle, the corrective order. The PIPC directed that the model Alipay built on the improperly obtained data be deleted. As the IAPP put it in its analysis, "model deletion eliminates the very asset an AI developer hopes to monetize." That is the whole point. A fine hits the balance sheet. A deletion order hits the product.

Why does an ocean-away Korean decision matter to US operators?

Because the logic travels, and because US regulators have already flirted with the same remedy. The Federal Trade Commission has used what it calls algorithmic disgorgement, ordering companies to delete not just wrongfully collected data but the models and algorithms derived from it, in matters like Everalbum and Rite Aid. What Korea did is the same idea, applied at scale, against a model sitting inside a payments product used by tens of millions. When two serious regulators on opposite sides of the world reach for the same tool, that tool is no longer theoretical. It is a live enforcement option that any counsel advising on AI needs to hold in view.

The practical exposure is this. If you cannot cleanly show that the data feeding a model was collected and transferred with a lawful basis, you are not just risking a fine on the data practice. You are risking the model that was built on top of it. That reframes a lot of diligence questions. It is no longer enough to ask whether a data source is useful. You have to ask whether, if a regulator later decides the source was tainted, the thing you built on it survives.

What should executives and counsel do now?

Treat training-data provenance as a first-class risk, documented and defensible, not a box a vendor checked. Know where each material training or fine-tuning dataset came from, what the lawful basis for its use was, and whether any of it involved a cross-border transfer that needed consent or disclosure. For anything acquired from a third party or another group entity, get the transfer basis in writing, because in this case the violation lived in exactly that seam between a Korean provider and an overseas affiliate. And build the muscle to answer a harder question than usual: if we were told to delete a model tomorrow, could we identify which datasets are in it, retrain without them, and prove we did. Companies that can answer that will treat a deletion order as a fire drill. Companies that cannot will treat it as a catastrophe.

Browse the full AI Regulation News tracker

Informational analysis for working professionals, not legal advice. Confirm how any enforcement action or data-protection requirement applies to your situation with qualified counsel in the relevant jurisdiction.