I believe very much in the most powerful and incorruptible regulatory regime possible for all business affairs, including AI. That regime called consumer regulation puts the power to enrich or destroy any business directly in the hands of the customers.
Individuals free to think and act according to their own reason and assessments will course-correct quickly and effectively. The freedom to buy and sell voluntarily are the most powerful and ultimately incorruptible regulatory force possible.
Customers are decentralized sovereign individuals, who can evaluate evidence independently. Each directly benefits from their own good judgement, or is harmed by poor decisions. This motivates quick error correction and optimization based on evidence. Knowledge spreads socially.
Customer assessments of benefit are immediately signalled in their buying decisions, the prices they are willing to bear, and their recommendations. Likewise, assessments of harm impair demand and reputation, which deny revenue to the bad supplier.
Contrast government regulation. Does not benefit from being right. Incurs no cost for being wrong. Has no liability for harm done to others for its poor decisions. Centralized control is corruptible, politicized, and often captured. Slow to react to new evidence or admit error.
Government power is limited to coercion, force, and penalties. Powerless against those who do not recognize its authority, such as criminals and black markets. Unlike the free market, where customers are equally powerful in all situations to buy and sell at will.
Conclusion: we must not usurp the most powerful regulatory power (customers) by delegating regulatory authority to government, which is weak, feckless, corruptible, dishonest, unaccountable, and a vulnerable regime that can be captured.
Thanks, @Mark_J_Perry, for this article which coins the term consumer-regulation.