OpenAI, the maker of ChatGPT, released new guidelines on Monday for assessing “catastrophic risks” from artificial intelligence in models in development. The news comes one month after the company’s board fired CEO Sam Altman, only to rehire him a few days later after staff and investors revolted.
According to US media, board members chastised Sam Altman for favoring OpenAI’s rapid development, even if it meant skirting certain questions about the technology’s potential risks.
What are OpenAI’s new guidelines to assess “catastrophic” AI risks?
The team will evaluate each new model and assign it a risk level, ranging from “low” to “critical,” in four categories.
According to the framework, only models with a risk score of “medium” or lower can be deployed.
The first category is concerned with cybersecurity and the model’s ability to conduct large-scale cyberattacks.
The second will assess the software’s proclivity to aid in the creation of a chemical mixture, an organism (such as a virus), or a nuclear weapon, all of which could endanger humans.
The third category is concerned with the model’s persuasive power, such as its ability to influence human behavior.
The final category of risk concerns the model’s potential autonomy, specifically whether it can escape the control of the programmers who created it.
Once the risks are identified, they will be forwarded to OpenAI’s Safety Advisory Group, a new body that will make recommendations to Sam Altman or someone appointed by him.
The head of OpenAI will then decide whether or not to make changes to a model to reduce the associated risks.
The board of directors will be kept informed and has the authority to overturn management decisions.