\n
![](\"https:\/\/content.fortune.com\/wp-content\/uploads\/2024\/02\/GettyImages-1241460440-e1709247167939.jpg?w=2048\")
<\/p>\n [ad_1]
\n<\/p>\n
Artificial intelligence software\u00a0doesn\u2019t always do<\/a> what the people building it want it to do\u00a0\u2014 a potentially dangerous issue that has consumed some of the largest companies working on the technology.<\/p>\n Big companies like\u00a0OpenAI<\/a>\u00a0and Alphabet Inc.\u2019s Google<\/a> are increasingly\u00a0directing workers<\/a>, money and computing power toward the problem. And Anthropic, an OpenAI competitor, has put it\u00a0at the heart<\/a>\u00a0of its development of Claude, a product it bills as a safer kind of AI chatbot.<\/p>\n Starting this month, a new company called\u00a0Synth Labs<\/a>\u00a0is also taking aim at the issue. Founded by a handful of prominent AI industry names, the company is emerging from stealth this week, and has raised seed funding from Microsoft<\/a> Corp.\u2019s venture capital fund, M12, and Eric Schmidt\u2019s First Spark Ventures. Synth Labs is primarily focused on building software, some of it open source, to help a range of companies ensure that their AI systems act according to their intentions. It\u2019s positioning itself as a company that\u2019s working transparently and collaboratively.<\/p>\n Alignment, as the issue is sometimes called, represents a technical challenge for AI applications such as chatbots that are built atop large language models, which are typically trained on huge swaths of internet data. The effort is complicated by the fact that people\u2019s ethics and values \u2014 as well as their ideas of what AI should and should not be permitted to do \u2014 vary. Synth Labs\u2019 products will aim to help steer and customize large language models, particularly models that are themselves\u00a0open source<\/a>.<\/p>\n The company got its start as a project within nonprofit AI research lab EleutherAI, worked on by two of the three founders \u2014 Louis Castricato and Nathan Lile \u2014 plus Synth Labs advisor and EleutherAI executive director Stella Biderman. Francis deSouza, former chief executive officer of biotechnology company Illumina<\/a> Inc., is also a founder. Synth Labs declined to say how much money it had raised so far.<\/p>\n Over the last few months the startup has built tools that can readily evaluate large language models on complex topics, Castricato said. The goal, he said, is to democratize access to easy-to-use tools that can automatically evaluate and align AI models.<\/p>\n A recent\u00a0research paper<\/a>\u00a0that Castricato, Lile and Biderman co-authored gives a sense for the company\u2019s approach: The authors used responses to prompts, which were generated by OpenAI\u2019s GPT-4 and Stability AI\u2019s Stable Beluga 2 AI models, to create a dataset. That dataset was then used as part of an automated process to direct a chatbot to avoid talking about one topic and instead talk about another.<\/p>\n \u201cThe way that we\u2019re thinking about designing some of these early tools really is all about giving you the opportunity to decide what alignment means for your business or your personal preferences,\u201d Lile said.<\/p>\n<\/div>\n\nSubscribe to the Eye on AI newsletter to stay abreast of how AI is shaping the future of business. Sign up<\/a> for free.<\/div>\n<\/div>\n