Devron was born in an environment where users and operators faced constraints due to siloed data across multiple jurisdictions, limited bandwidth, and high data transfer infrastructure overhead. This situation necessitated a solution enabling computing, analytics, and decision-making capabilities at the location where data was collected, thereby showcasing the power of federated learning.
Devron is a pioneer in federated cloud-based data science and machine learning. Our platform unlocks access to private data stored globally and across various clouds and mobile computing environments.
When it comes to generative AI and large language models (LLMs), Devron is driving innovation by making it easier to share context and fine-tune models on siloed datasets.
LLMs are foundational models trained using vast amounts of text data, with some models encompassing as many as 300 billion words and at least 1 billion parameters. They can comprehend context, infer meanings, and generate coherent and contextually relevant text closely resembling human communication.
LLMs find applications in various tasks, including language translation, content generation, text summarization, and question answering.
Context Matters
Context plays a pivotal role in human communication and comprehension, and this significance is equally applicable to LLMs. LLMs like GPT-4, LLaMA, and BARD have made remarkable strides in generating human-like text and performing various natural language processing tasks. However, their effectiveness heavily relies on understanding and incorporating context. It provides the framework that allows these models to generate relevant, coherent, and accurate responses.
Context refers to the surrounding information, ideas, or circumstances that give meaning to a particular piece of text or conversation. It includes both the immediate linguistic context (preceding and following words or sentences) and the broader situational or topical context. Incorporating context effectively allows LLMs to produce responses that align with the ongoing discussion.
For enterprise buyers, sharing context within departments can be transformative. For example, it can allow some users to engage with an LLM model about briefs and get information on legal summaries, while others get undergarments. This contextual awareness significantly improves the quality of insights and responses the LLM provides, fostering improved cross-department collaboration and decision-making.
The Challenge of Building LLMs for Enterprises
Training large language models for enterprises poses numerous challenges.
Data is spread across different departments, repositories, and devices in an enterprise environment. To train the model, typically, organizations must bring the data together in one place. However, transferring or duplicating this data becomes increasingly complex, costly, and time-consuming due to its volume, cloud distribution, regulatory or data sovereignty restrictions, and privacy and data leakage risks.
Devron overcomes these challenges by leaving the data where it resides and bringing the model to the data. By enabling the training of an LLM without moving data, Devron can reduce the overhead, cost, and risk of transporting and replicating data—thereby accelerating model development and insights.
In addition, within a large organization, different departments often hold different types of data. For instance, the marketing department might have information about customer preferences and purchase patterns. In contrast, the legal and compliance department might have data about regulation and compliance issues, and the fraud or risk department might have data about fraudulent activities and patterns. This data is often highly sensitive and cannot be shared, even across departments, due to privacy and compliance regulations.
Devron’s federated learning platform enables departments to securely share their data for model training without compromising data security or privacy, as only insights—not raw data—are shared across the enterprise.
Build Better LLMS with Fine-Tuning, In-Context Learning, & Multi-Cloud Federated ML
We see a world where enterprises use a mix of fine-tuning and in-context learning to help LLMs better serve the needs of departments in their organization.
Fine-Tuning
Devron can fine-tune LLMs, like GPT-3 or GPT-4 hosted by OpenAI or host a set of models (like Alpaca) within an enterprise, each of which is trained (or fine-tuned) on a specific domain and accessible to users in departments who have access to the knowledge in that domain.
Devron’s APIs enable these models to be imported using HuggingFace, or other CloudML platforms like SageMaker, AzureML Studio, or Vertex AI.
In-Context Learning
Devron provides a central service (called “Control Center”) that can inject context for user queries based on the information accessible to a user. In addition, Devron’s Privacy Portal enables data owners to provision data to Devron’s federated ecosystem as the built-in trust and safety layer.
The Control Center will enforce authorization and authentication, crawl and index enterprise content (often stored in customers’ flat file storage buckets like S3 or similar), integrate with vector databases to provide semantic search capabilities, and select and use the suitable embeddings model and LLMs—vastly lowering the barrier of entry to integrate LLMs into enterprise use cases.
Multi-Cloud Federated Machine Learning
Each data source can be from different departments, devices, jurisdictions, etc., providing users with flexibility and usability across diverse datasets. Devron facilitates contributions from multiple unique datasets in various locations, concurrently training against them to yield a model. This approach addresses the cost of organizing and grouping all the data required for training these LLMs, which is often half the battle.
Thanks to Devron's multi-cloud and modular architecture, accessing and concurrently processing the data needed to train these models where the data resides is possible.
A Devron deployment consists of a Control Center and one or more Satellites. Data scientists develop models and structure training jobs at the Control Center, then submit them to Satellites. Each Satellite connects to and processes data from proximate data sources in the same cloud account and jurisdiction, regardless of whether they are from separate devices, systems, or formats. All the training and instruction execution happens at these Satellites.
Once a model has been trained against designated datasets, each Satellite sends back encrypted weights and parameters—but not data—to the Control Center. Using Devron APIs and other instructions, the Control Center aggregates the model weights and parameters and then sends the updated model back to the Satellite for continued training. This process continues until the model converges.
Drive Enterprise Innovation with Devron
Building an LLM requires massive amounts of data, and gathering that together in one location is time-consuming, expensive, and risky—especially if you’re an enterprise organization. Not to mention, enterprise data can be highly sensitive, making it difficult or impossible to share.
Devron solves both these issues by leaving the data in situ—safe and secure. The benefits are three-fold: (1) acceleration of model development by not moving data, (2) lower costs of data transfer, and (3) preservation of data privacy.
In addition, Devron’s ability to fine-tune LLM models and provide in-context learning results in more personalized enterprise models.
If you’re evaluating how to best leverage large language models in your organization, including building your own large language model, but you’re running into challenges regarding context or distribution and privacy restrictions of your data, contact us to learn how we can accelerate your efforts, save money, and drive greater innovation.
