After making its debut with Bard and the Pixel 8 Pro a week ago, Gemini, the latest flagship gen AI model family unveiled by Google, is now available to Google Cloud customers through the Vertex AI platform.
Gemini Pro on Vertex AI
Google has announced the release of Gemini Pro on Vertex AI, a lightweight version of the more advanced Gemini Ultra model. While the latter is currently only available in private preview, Gemini Pro is now accessible in public preview via the new Gemini Pro API. The API can be used for free, but with certain limitations for the time being. It supports 38 languages and regions, including Europe, and comes with features such as chat functionality and filtering.
“Gemini is a state-of-the-art, natively multimodal model with advanced reasoning and coding skills,” said Thomas Kurian, CEO of Google Cloud, during a press briefing on Tuesday. “Now, developers can build their own applications using this powerful tool.”
The Gemini Pro API
The Gemini Pro API in Vertex is primarily designed to accept text as input and generate text as output, similar to other generative text model APIs such as Anthropic’s, AI21’s, and Cohere’s. However, there is also an additional endpoint available, called Gemini Pro Vision. This feature, launching today in preview, can process both text and imagery, including photos and videos, and generate text using a model similar to OpenAI’s GPT-4 with Vision.
The addition of image processing addresses one of the main criticisms surrounding Gemini since its release last Wednesday. The previous version of Gemini, known as the Gemini Pro model, used by Bard, was unable to process images, despite being marketed as a “multimodal” model, trained on a variety of data including text, images, videos, and audio. With the new Gemini Pro API, users now have the option to test the model’s image analysis capabilities for themselves.
Within Vertex AI, developers can customize Gemini Pro to specific contexts and use cases, using the same fine-tuning tools available for other models hosted on Vertex. Additionally, Gemini Pro can be connected to external APIs to perform specific actions or “grounded” for improved accuracy and relevance of responses. This grounding can be done using third-party data from an app or database, or from the web and Google Search.
Citation checking, another existing capability in Vertex AI, is now also available for Gemini Pro models. This feature serves as an additional fact-checking measure by highlighting the sources of information used by the model to generate a response.
“Grounding allows us to take an answer generated by Gemini and compare it with a set of data from a company’s own systems or web sources,” explained Kurian. “This comparison helps improve the quality of the model’s answers.”
Kurian also emphasized the control, moderation, and governance options available with Gemini Pro, seemingly addressing criticism that the model may not be the strongest out there. To further entice developers, Google is offering discounts for using Gemini Pro on Vertex AI.
Input for Gemini Pro on Vertex AI is priced at $0.00025 per character, while output costs $0.00005 per character. These prices have been reduced by 4x and 2x, respectively, from the previous version of Gemini Pro. For a limited time, until early next year, the use of Gemini Pro on Vertex AI is free for customers.
“Our goal is to attract developers with attractive pricing,” Kurian stated.
Strengthening Vertex AI
Apart from the launch of Gemini Pro, Google has also introduced several new features to Vertex AI to stay competitive with other platforms such as Bedrock.
Some of these features are specifically related to Gemini Pro. Soon, Vertex customers will be able to use Gemini Pro to power custom-built conversational voice and chat agents, allowing for dynamic interactions with advanced reasoning capabilities. Gemini Pro will also be an option for search summarization, recommendations, and answer generation in Vertex AI, leveraging data from various modalities and sources.
According to Kurian, these conversational and search features powered by Gemini Pro are expected to be available “very early” in 2024.
In addition to these enhancements, Google has introduced Automatic Side-by-Side (Auto SxS) for model evaluation, which is considered faster and more cost-efficient than manual evaluations. Auto SxS is an answer to AWS’ recently launched Model Evaluation feature on Bedrock.
Google is also collaborating with third-party models, such as Mistral and Meta, and introducing “step-by-step” distillation, a technique that creates smaller and more specialized models for lower latency from larger models. To further reassure enterprise customers, Google’s indemnification policy has been extended to include outputs from PaLM 2 and Imagen models.
Some vendors using generative AI models have faced criticism for regurgitating training data, causing concerns for corporate customers. To address these concerns, Google has expanded its indemnification policy for third-party models. Although it has not yet been extended to cover the Gemini Pro API, Google says it will do so upon its public release.
“Our goal is to provide reassurance to developers using our products,” added Kurian.
