The world of artificial intelligence is abuzz with the rising popularity of text-generating AI, commonly referred to as large language models (LLMs). These models, such as ChatGPT, are becoming a top priority for enterprise organizations, with a recent survey showing that 67.2% of companies plan to adopt LLMs by early 2024.
However, there are significant barriers blocking the adoption of LLMs. The survey also revealed that a lack of customization and flexibility, along with concerns about preserving company knowledge and intellectual property, are causing hesitation among businesses looking to deploy LLMs in production.
This dilemma sparked the curiosity of Varun Vummadi and Esha Manideep Dinne, leading them to found Giga ML. Their startup is focused on developing a platform that allows companies to deploy LLMs locally, reducing costs and preserving privacy in the process.
“Data privacy and customization are the biggest challenges enterprises face when adopting LLMs. Giga ML aims to solve both of these obstacles,” Vummadi shared in an email interview with TechCrunch.
Giga ML offers its own set of LLMs, known as the “X1 series,” designed for tasks like generating code and answering common customer questions. The models, built on top of Meta’s Llama 2, have shown to outperform other popular LLMs on certain benchmarks, such as the MT-Bench test set for dialogs. However, it is difficult to determine the qualitative comparison of X1 due to technical issues encountered when testing the online demo.
Despite the potential superiority of Giga ML’s models, can they truly stand out in the crowded market of open source and offline LLMs?
In a conversation with Vummadi, it became clear that Giga ML’s goal is not to create the best-performing LLMs, but rather to provide tools for businesses to fine-tune LLMs locally without relying on third-party resources and platforms.
“Giga ML’s mission is to help enterprises securely and efficiently deploy LLMs on their own on-premises infrastructure or virtual private cloud. We simplify the process of training, fine-tuning, and running LLMs with an easy-to-use API, eliminating any associated hassle,” Vummadi explained.
Vummadi emphasized the privacy benefits of running models offline, an appealing factor for many businesses. A survey by low-code AI dev platform Predibase revealed that less than a quarter of enterprises are comfortable using commercial LLMs due to concerns about sharing sensitive and proprietary data with vendors. The majority of respondents (77%) do not use nor plan to use commercial LLMs in production due to issues surrounding privacy, cost, and customization.
“IT managers at the C-suite level find Giga ML’s offerings valuable because of the secure on-premise deployment of LLMs, customizable models tailored to their specific use case, and fast inference, which ensures data compliance and maximum efficiency,” Vummadi stated.
To support its growing customer base, Giga ML has raised ~$3.74 million in VC funding from Nexus Venture Partners, Y Combinator, Liquid 2 Ventures, 8vdx, and others. The startup plans to expand its two-person team and focus on product R&D in the near future. Currently, Giga ML’s customers consist of finance and healthcare companies, although their names have not been disclosed.
