California SB 574: AI is entering legal practice

California has passed SB574, a bill governing how lawyers, arbitrators and parts of the justice system may use Generative AI.

For me, this is another very interesting example of law trying to catch up with technological reality. We often discuss whether AI is still hype or whether it has already become part of normal professional work. AI is already embedded in real business and professional processes, and regulators are now responding to the risks created by that use.

Photo: magnific

Lawyers across the US are increasingly using AI tools, while courts have already sanctioned parties for submitting false or fabricated AI-generated material without proper verification. Under SB 574, an attorney must not delegate the practice of law to generative AI. When AI assists legal work, the lawyer remains responsible for the final result.

The bill introduces several requirements:

➡️ lawyers must take reasonable steps to verify AI-generated outputs, including case and statutory citations;

➡️ erroneous or hallucinated content must be corrected before it is used;

➡️ the use of generative AI must be disclosed to the court for documents submitted in proceedings;

➡️ confidential, personally identifying and other non-public information cannot be entered into an AI system unless access to that information is appropriately restricted;

➡️ court filings cannot contain citations that the responsible attorney has not personally verified;

➡️ arbitrators cannot delegate any part of their decision-making process to AI.

The bill explicitly refers to sensitive information such as medical or psychiatric data, financial information, account numbers, Social Security numbers, dates of birth, addresses and other confidential content. So, it is also a Data Protection, confidentiality, Compliance and professional-accountability issue.

In the EU, we have the AI Act as a horizontal regulatory framework. In the US, we continue to see a legal patchwork.

What I also find interesting is how quickly AI is becoming ordinary outside heavily regulated sectors. Recently, my parents told me their hairdresser uses AI to show clients how they might look with different hairstyles. AI is becoming part of everyday work in legal services, healthcare, finance, software development, marketing, and even hair salons. So when we ask whether AI is still hype, regulation increasingly gives us the answer. You do not create professional rules around a technology nobody uses.

Author: Sebastian Burgemejster