Re Geoffrey Hinton’s concerns about the perils of artificial intelligence (‘Godfather of AI’ shortens odds of the technology wiping out humanity over next 30 years, 27 December), I believe these concerns can best be mitigated through collaborative research on AI safety, with a role for regulators at the table.
Currently, frontier AI is tested post-development using “red teams” who try their best to elicit a negative outcome. This approach will never be enough; AI needs to be designed for safety and evaluation – something that can be done by drawing on expertise and experience in well-established safety-related industries.
Hinton does not seem to think that the existential threat from AI is one which is deliberately being encoded – so why not enforce the deliberate avoidance of this scenario? While I don’t subscribe to his perspective about the level of risk facing humanity, the precautionary principle suggests that we must act now.
In traditional safety-critical domains, the need to build physical systems, eg aircraft, limits the rate at which safety can be impacted. Frontier AI has no such physical “rate-limiter” on deployment, and this is where regulation needs to play a role. Ideally, there should be a risk assessment prior to deployment but the current risk metrics are inadequate – for example, they don’t consider the application sector, or scale of deployment.
Regulators need the power to “recall” deployed models (and the big companies that develop them need to include mechanisms to stop particular uses) as well as supporting work on risk assessment, which will give leading indicators of risk, not just lagging indicators. Put another way, the government needs to focus on post-market regulatory controls while supporting research that enables regulators to have the insights to enforce pre-market controls. This is challenging, but imperative if Hinton is right about the level of risk facing humanity.
Prof John McDermid
Institute for Safe Autonomy, University of York
Source link
lol
