Goodfire raises $7M for its ‘brain surgery’-like AI observability platform

Goodfire, a startup developing tools to increase observability of the inner workings of generative AI models, announced today that it has raised $7 million in seed funding led by Lightspeed Venture Partners, with participation from Menlo Ventures, South Park Commons, Work-Bench, Juniper Ventures, Mythos Ventures, Bluebirds Capital, and several notable angel investors.

Addressing the ‘black box’ problem

As generative AI models like large language models (LLMs) become increasingly complex — with hundreds of billions of parameters, or internal settings governing their behavior — they have also become more opaque.

This “black box” nature poses significant challenges for developers and businesses looking to deploy AI safely and reliably.

A 2024 McKinsey survey highlighted the urgency of this problem, revealing that 44% of business leaders have experienced at least one negative consequence due to unintended model behavior.

Goodfire aims to address these challenges by leveraging a novel approach called “mechanistic interpretability.”

This field of study focuses on understanding how AI models reason and make decisions at a detailed level.

Editing model behavior?

Goodfire’s product is pioneering the use of interpretability-based tools for understanding and editing AI model behavior. Eric Ho, CEO and co-founder of Goodfire, explains their approach:

“Our tools break down the black box of generative AI models, providing a human-interpretable interface that explains the inner decision-making process behind a model’s output,” Ho said in an emailed response to VentureBeat. “Developers can directly access the inner mechanisms of the model and change how important different concepts are to modify the model’s decision-making process.”

The process, as Ho describes it, is akin to performing brain surgery on AI models. He outlines three key steps:

1. Mapping the brain: “Just as a neuroscientist would use imaging techniques to see inside a human brain, we use interpretability techniques to understand which neurons correspond to different tasks, concepts, and decisions.”
2. Visualizing behavior: “After mapping the brain, we provide tools to understand which pieces of the brain are responsible for problematic behavior by creating an interface that lets developers easily find problems with their model.”
3. Performing surgery: “With this understanding, users can make very precise changes to the model. They might remove or enhance a specific feature to correct model behavior, much like a neurosurgeon might carefully manipulate a specific brain area. By doing so, users can improve capabilities of the model, remove problems, and fix bugs.”

This level of insight and control could potentially reduce the need for expensive retraining or trial-and-error prompt engineering, making AI development more efficient and predictable.

Building a world-class team

The Goodfire team brings together experts in AI interpretability and startup scaling:

• Eric Ho, CEO, previously founded RippleMatch, a Series B AI recruiting startup backed by Goldman Sachs.
• Tom McGrath, Chief Scientist, was formerly a senior research scientist at DeepMind, where he founded the company’s mechanistic interpretability team.
• Dan Balsam, CTO, was the founding engineer at RippleMatch, where he led the core platform and machine learning teams.

Nick Cammarata, a leading interpretability researcher formerly at OpenAI, emphasized the importance of Goodfire’s work: “There is a critical gap right now between frontier research and practical usage of interpretability methods. The Goodfire team is the best team to bridge that gap.”

Nnamdi Iregbulem, Partner at Lightspeed Venture Partners, expressed confidence in Goodfire’s potential: “Interpretability is emerging as a crucial building block in AI. Goodfire’s tools will serve as a fundamental primitive in LLM development, opening up the ability for developers to interact with models in entirely new ways. We’re backing Goodfire to lead this critical layer of the AI stack.”

Looking ahead

Goodfire plans to use the funding to scale up its engineering and research team, as well as enhance its core technology.

The company aims to support the largest state-of-the-art open weight models available, refine its model editing functionality, and develop novel user interfaces for interacting with model internals.

As a public benefit corporation, Goodfire is committed to advancing humanity’s understanding of advanced AI systems. The company believes that by making AI models more interpretable and editable, they can pave the way for safer, more reliable, and more beneficial AI technologies.

Goodfire is actively recruiting “agentic, mission-driven, kind, and thoughtful people” to join their team and help build the future of AI interpretability.