CalypsoAI Raises $23M To Add Guardrails To Generative AI Models

As the adoption of generative AI continues to increase among companies, it becomes crucial to address the associated blockers and challenges that arise when incorporating this technology into existing workflows. Many businesses have concerns and doubts about how to effectively mitigate the risks involved. Recognizing these concerns, Neil Sereybrany founded CalypsoAI, a startup focused on developing software that enables testing, validation, and monitoring of internally-developed and third-party AI applications before their deployment.

Sereybrany emphasizes that CalypsoAI’s platform is designed to harness the power of AI while ensuring a safe, secure, and reliable environment. By addressing these critical aspects, the company aims to provide organizations with the competitive advantage that AI offers. The confidence in CalypsoAI’s capabilities is reflected in the recent funding announcement. The startup has successfully raised $23 million in a Series A-1 funding round led by Paladin Capital Group, with participation from notable investors such as Lockheed Martin Ventures, Hakluyt Capital, and Expeditions Fund.

The funding round also saw contributions from notable angels, including former YouTube CEO Susan Wojcicki and 23andMe co-founder Anne Wojcicki, further validating the potential of CalypsoAI. With this latest funding, the startup has now raised a total of $38.2 million. The support from investors underscores the belief in CalypsoAI’s mission and its ability to deliver effective solutions in the field of AI governance and risk mitigation.

CalypsoAI’s comprehensive approach to addressing the challenges associated with generative AI has resonated with investors and industry experts alike. The startup’s focus on ensuring the safe and secure deployment of AI applications positions it as a key player in the rapidly evolving AI landscape. With its continued growth and commitment to innovation, CalypsoAI aims to carve a significant path in the market while helping companies harness the power of AI with confidence and reliability.

CalypsoAI offers a range of tools that can be seamlessly deployed within an organization’s infrastructure, acting as a container to monitor and shape the usage of large language models, such as ChatGPT. Through intuitive dashboards, businesses can access statistics related to model toxicity, user engagement, and more. Importantly, CalypsoAI’s technology safeguards sensitive company data, preventing its inadvertent sharing within models, while also identifying and mitigating attacks originating from generative AI tools.

In the corporate sector, there is a clear demand for solutions that provide guardrails for generative AI. Concerns about accidental exposure of proprietary information have led companies like Apple, JPMorgan Chase, and Verizon to ban or restrict the use of ChatGPT-like tools internally. Furthermore, these organizations are cautious about the issues of toxicity and misinformation associated with generative AI, which persist despite advancements in model quality.

Recognizing the vast market potential, industry players have risen to meet this demand. Microsoft’s Azure OpenAI Service offers governance and compliance capabilities atop generative AI models, including ChatGPT, from OpenAI. Additionally, Salesforce has recently introduced the Einstein Trust Layer, aiming to prevent text-generating models from retaining sensitive data.

However, the effectiveness of such guardrails in practice remains an open question. Recently, researchers uncovered flaws in Nvidia’s NeMo Guardrails toolkit, designed to enhance the accuracy and security of AI-powered applications, allowing for the bypassing of its protections.

Addressing these concerns, Neil Sereybrany emphasizes that CalypsoAI’s products are robust against attacks. He also emphasizes that the company does not retain any customer data but rather facilitates the flow of information from employees into the chosen model of the organization.

Securing large language models (LLMs) in the enterprise has become a critical endeavour, leading to CalypsoAI’s strategic shift to meet the needs of enterprises seeking to leverage AI solutions securely. Sereybrany expresses confidence in the company’s ability to develop exceptional products and attract top talent to ensure the security of enterprise organizations. As organizations increasingly seek secure and reliable generative AI solutions to drive their businesses forward, CalypsoAI aims to deliver the best possible outcomes while prioritizing data protection and organizational security.

With the newly raised funds, CalypsoAI intends to allocate resources towards product development, expanding its workforce, and advancing its go-to-market strategies. Currently employing 35 individuals, the startup has recently established an office in Dublin, aligning with its objective of doubling its workforce within the next two years.

“As our company continues to experience growth and expansion, we acknowledge the significance of partnering with new investors to bolster our progress,” Sereybrany emphasized. “By collaborating with investors who share our vision and goals, we can sustain innovation and propel our business to new heights.”