Cohere, a startup developing large-scale language models that rival those of OpenAI, today announced the creation of a non-profit research lab: Cohere For AI. Led by Google graduate Sara Hooker, Cohere says Cohere Labs will work to solve some of the industry’s toughest challenges by bringing “fundamental research” to the open source community.
“We’re excited to lead a new nonprofit AI research lab as we continue to expand how and where research is done. There is so much to discover and our focus will be on open collaboration and contributing to basic research,” Hooker told TechCrunch via email. “At the same time, a key component of our work will be to grow the community and help educate the next generation of talent and create new entry points for working on core research. “
There has long been concern in the AI community that there is insufficient funding for AI research outside of wealthy corporations. One study found that ties to companies — either funding or affiliation — increased significantly in AI research from 2008 to 2019. Another study showed that Google’s parent company Alphabet, along with Amazon and Microsoft, hired a whopping 52 tenure-track AI professors between 2004 and 2018 in a bid to remove these would-be teachers from academic and nonprofit work.
The concentration of power within companies has a number of obvious downsides, but one of the most alarming is that values such as charity, equity and inclusion tend to be undervalued on the research side. A number of experts speaking to Wired for a 2020 article point out that corporate AI projects have led to an “unscientific fixation” on projects only possible for people with access to powerful data centers. Whatever the domain, corporate work is often closely guarded and takes years to see the light of day—if it ever does.
“Our agenda is focused on driving progress on machine learning issues alongside community-focused research,” Hooker said. “We also want to have a proactive research agenda so we can identify big challenges before they become problems that we need to fix retrospectively. For example, we focus on a variety of different disciplines to work on mitigating bias, and a core research area is AI safety and robust use of models.”
Another core component Cohere For AI hopes to expand is access to computing resources, Hooker said — specifically to help researchers better leverage “state-of-the-art” models to develop their work.” The role of computing access is changing, as is illustrate the trends in language models (ie AI systems that understand and generate text). Just a few years ago, creating a sophisticated language model required enormous computational resources. But now, thanks to academic breakthroughs and the work of the open source community, the barriers to entry are much lower than they used to be.
path to charity
Backed by AI luminaries including Pieter Abbeel, co-director of UC Berkeley’s AI lab, Cohere was founded in 2019 by a pedigree team that includes Aidan Gomez, Ivan Zhang and Nick Frosst. Gomez co-authored the academic paper Attention Is All You Need, which introduced the world to a fundamental AI model architecture called Transformer. (Among other high-profile systems, OpenAI’s GPT-3 and Codex are based on the Transformer architecture.) Zhang is a collaborator alongside Gomez at For.ai, an open AI research collective that includes data scientists and engineers.
“For.ai was designed to help early-career enthusiasts better connect with more experienced researchers,” Hooker said. “Many of the founding members have PhDs or worked in academic or industrial laboratories. At the time, For.ai was one of the first community-driven research groups supporting independent researchers around the world. Now the Cohere team and its supporters look forward to reintroducing the original concept but with more resources built on top of Cohere.”
According to Hooker, Cohere For AI will provide opportunities for data scientists to meet and collaborate through mentoring research opportunities, attendance at traditional conferences and contributions to research journals. Part of this is done by promoting the stewardship of open source scientific practices and the “responsible” release of code as well Supporting efforts that promote “scientific communication” through various mediums such as blog posts.
“WWe really want to design Cohere For AI as an ambitious research lab that contributes to the research community but also prioritizes to better include a variety of voices. We want to help change where, how and by whom research is done,” Hooker said.
Despite its lofty goals, Cohere For AI – which Cohere will fund itself – is likely to provoke skepticism among researchers wary of Cohere’s corporate relationships. Cohere has raised $170 million to date from institutional venture capital firms such as Tiger Global Management and Index Ventures, and has a number of ties to Google. Google Cloud AI Chief Scientist Fei-Fei Li and Google Fellow Geoffrey Hinton were early supporters of Cohere, and Gomez and Frosst previously worked at Google Brain, one of Google’s AI research divisions. Cohere also has a partnership with Google to train large language models on the company’s dedicated hardware infrastructure.
Google dissolved an AI advisory board in 2019, just a week after its inception. And in 2020, the company fired leading AI researcher Timnit Gebru in retaliation for emailing colleagues criticizing Google’s management practices. Google then fired another ethicist, Margaret Mitchell, who had publicly denounced the company’s handling of the situation, and a third, Satrajit Chatterjee, after contributing to a paper questioning Google’s work on AI-assisted chip design systems .
Paved with good intentions
Broadly speaking, nonprofit initiatives funding AI research are mixed.
Success stories include the Allen Institute for AI (AI2), founded by the late Microsoft co-founder Paul Allen, which seeks scientific breakthroughs by constructing AI systems with logic capabilities. While not strictly non-profit, Anthropic, launched by former OpenAI executives, has raised over half a billion dollars to research “reliable, interpretable, and controllable” AI systems.
But for every AI2 and Anthropic, there’s an OpenAI that started out as a non-profit organization before transitioning to limited profit and accepting a $1 billion investment from Microsoft. Former Google Chairman Eric Schmidt’s recently announced $125 million fund for AI research sparked new controversy after Politico reported that Schmidt wields unusually strong influence over the White House Office of Science and Technology Policy. (One of the early recipients, Berkeley professor Rediet Abebe, requested that her name be removed from consideration.)
However, some recent collectives have shown promise, most notably Gebrus Distributed AI Research, a global AI research nonprofit. Projects like Hugging Face’s BigScience and EleutherAI are other strong examples of what can be achieved with AI beyond the confines of corporate influence.
“Ultimately, it’s up to us to prove that Cohere For AI won’t turn to the gamble over time,” Hooker said. “Although Cohere For AI will depend on Cohere for resources and funding, a clear separation between the two has been created to maintain its independence as a research laboratory. This separation is critical for her to continue to contribute and serve as an independent entity to the broader community. Structured as a non-profit organization, Cohere For AI was intentionally designed to work openly with many different organizations. His work will be open source to give the wider community more access.”