Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the thegem domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/j8p72agj2cgw/fanaticalfuturist.com/wp-includes/functions.php on line 6121

Notice: Function _load_textdomain_just_in_time was called incorrectly. Translation loading for the wp-2fa domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home/j8p72agj2cgw/fanaticalfuturist.com/wp-includes/functions.php on line 6121
First of a kind AI that gives data owners full control of their data emerges – Matthew Griffin | Keynote Speaker & Master Futurist
Scroll Top

First of a kind AI that gives data owners full control of their data emerges

WHY THIS MATTERS IN BRIEF

Most AIs are trained using huge volumes of third party data and the data owners have no control over its use or removal from models, this changes that.

 

Love the Exponential Future? Join our XPotential Community, future proof yourself with courses from XPotential University, read about exponential tech and trendsconnect, watch a keynote, or browse my blog.

A new kind of Large Language Model (LLM), developed by researchers at the Allen Institute for AI (Ai2), makes it possible to control how training data is used even after a model has been built.

 

RELATED
Jailbreaking the world's most advanced LLM based robots is super easy

 

The new model, called FlexOlmo, could challenge the current industry paradigm of big Artificial Intelligence (AI) companies slurping up data from the web, books, and other sources – often with little regard for ownership – and then owning the resulting models entirely. Once data is baked into an AI model today, extracting it from that model is a bit like trying to recover the eggs from a finished cake.

“Conventionally, your data is either in or out,” says Ali Farhadi, CEO of Ai2, based in Seattle, Washington. “Once I train on that data, you lose control. And you have no way out, unless you force me to go through another multi-million-dollar round of training.”

Ai2’s avant-garde approach divides up training so that data owners can exert control. Those who want to contribute data to a FlexOlmo model can do so by first copying a publicly shared model known as the “anchor.” They then train a second model using their own data, combine the result with the anchor model, and contribute the result back to whoever is building the third and final model.

 

RELATED
Experts: AI 'will beat humans in all tasks by 2060, and take all jobs by 2140'

 

Contributing in this way means that the data itself never has to be handed over. And because of how the data owner’s model is merged with the final one, it is possible to extract the data later on. A magazine publisher might, for instance, contribute text from its archive of articles to a model but later remove the sub-model trained on that data if there is a legal dispute or if the company objects to how a model is being used.

“The training is completely asynchronous,” says Sewon Min, a research scientist at Ai2 who led the technical work. “Data owners do not have to coordinate, and the training can be done completely independently.”

The FlexOlmo model architecture is what’s known as a Mixture of Experts a popular design that is normally used to simultaneously combine several sub-models into a bigger, more capable one. A key innovation from Ai2 is a way of merging sub-models that were trained independently. This is achieved using a new scheme for representing the values in a model so that its abilities can be merged with others when the final combined model is run.

 

RELATED
The Creative Machines are coming

 

To test the approach, the FlexOlmo researchers created a dataset they call Flexmix from a mix of proprietary and public data designed to simulate closed datasets. They used the FlexOlmo design to build a model with 37 billion parameters, about a tenth of the size of the largest open source model from Meta. They then compared their model to several others. They found that it outperformed any individual model on all tasks and also scored 10 percent better at common benchmarks than two other approaches for merging independently trained models.

The result is a way to have your cake – and get your eggs back, too.

“You could just opt out of the system without any major damage and inference time,” Farhadi says. “It’s a whole new way of thinking about how to train these models.”

Percy Liang, an AI researcher at Stanford, says the Ai2 approach seems like a promising idea.

“Providing more modular control over data – especially without retraining – is a refreshing direction that challenges the status quo of thinking of language models as monolithic black boxes,” he says. “Openness of the development process – how the model was built, what experiments were run, how decisions were made – is something that’s missing.”

 

RELATED
Researchers trained an OpenAI rival in half an hour for $50

 

Farhadi and Min say that the FlexOlmo approach might also make it possible for AI firms to access sensitive private data in a more controlled way, because that data does not need to be disclosed in order to build the final model. However, they warn that it may be possible to reconstruct data from the final model, so a technique like Differential Privacy, which allows data to be contributed with mathematically guaranteed privacy, might be required to ensure data is kept safe.

Ownership of the data used to train large AI models has become a big legal issue in recent years. Some publishers are suing large AI companies while others are cutting deals to grant access to their content.

In June, Meta won a major copyright infringement case when a federal judge ruled that the company did not violate the law by training its open source model on text from books by 13 authors.

Min says it may well be possible to build new kinds of open models using the FlexOlmo approach.

 

RELATED
MIT researchers can now use lasers to look at your insides from a distance

 

“I really think the data is the bottleneck in building the state of the art models,” she says. “This could be a way to have better shared models where different data owners can codevelop, and they don’t have to sacrifice their data privacy or control.”

Related Posts

Leave a comment

Pin It on Pinterest

Share This