Zoho Teams Up with NVIDIA for AI Models

Zoho Corporation, a global technology firm based in Chennai, has announced its plan to utilize NVIDIA’s AI-accelerated computing platform, including NVIDIA NeMo as part of NVIDIA AI Enterprise software, to develop and deploy large language models (LLMs) within its SaaS applications. These LLMs will be accessible to Zoho’s more than 700,000 customers worldwide across ManageEngine and Zoho.com.

Over the past year, Zoho has invested over USD 10 million in NVIDIA’s AI technology and GPUs, with plans to invest an additional USD 10 million in the upcoming year. This strategic announcement was made during the NVIDIA AI Summit in Mumbai.

Ramprakash Ramamoorthy, Director of AI at Zoho Corporation, emphasized the company’s commitment to creating LLMs specifically designed for business applications. “Many LLMs currently available focus on consumer use and offer limited value for businesses. At Zoho, we aim to develop models tailored for diverse business scenarios. By owning our entire tech stack, we can integrate the essential element that makes AI truly effective: context,” he stated.

Zoho prioritizes user privacy from the start, ensuring its models are compliant with privacy regulations. The company aims to help businesses achieve rapid ROI by fully utilizing NVIDIA’s AI software and accelerated computing to enhance throughput and reduce latency.

With over a decade of experience in building its own AI technologies, Zoho integrates contextual intelligence into its extensive portfolio of over 100 products across ManageEngine and Zoho divisions. The company’s approach involves creating narrow, small, and medium language models, providing flexible options for various use cases. This model diversity allows businesses with limited data to still benefit from AI solutions, while ensuring customer data remains private—core to Zoho’s AI strategy.

“The ability to choose from a range of AI model sizes empowers businesses to tailor their AI solutions precisely to their needs, balancing performance with cost-effectiveness,” said Vishal Dhupar, Managing Director, Asia South at NVIDIA. “With NVIDIA’s AI software and accelerated computing platform, Zoho is building a broad range of models to serve the diverse needs of its business customers.”

Through this collaboration, Zoho will enhance its LLMs on the NVIDIA accelerated computing platform with NVIDIA Hopper GPUs, employing the NVIDIA NeMo platform for developing custom generative AI solutions—including LLMs, multimodal, vision, and speech AI. Zoho is also testing NVIDIA TensorRT-LLM to optimize its LLMs for deployment, reporting a 60% increase in throughput and a 35% reduction in latency compared to its previous open-source framework. Additionally, the company is accelerating other workloads like speech-to-text on NVIDIA’s advanced computing infrastructure. 

This initiative marks a significant step for Zoho as it continues to innovate in the AI space while maintaining a strong focus on privacy and tailored business solutions.