May 24, 2023, 03:30 am, – Updated May 24, 2023, 04:21 am
The language class continues to be the most accurate and proven, and while OpenAI seems to be leading with GPT-4, others — and not just Open Source — are advancing. This is demonstrated by Google’s PaLM II model, but also by Meta’s (Facebook) LLaMa model, which now has promising variants called LIMA.
Two very different workouts. As a recent Meta study explains, large samples are learned at two levels of language. In the first, there is no retrospective training that starts from the raw text and that learns in the general purpose of the mission. In the second, debugging of these models is carried out, and reinforcement learning is applied to align the model with certain services or user preferences.
SHOES This is exactly what Meta has done by establishing and implementing LIMA (Less Is More for Alingment), a language model based on LLaMa with 65 million parameters and which is finely tuned with only 1,000 suggestions and responses specially prepared to behave appropriately. . No reinforcement was necessary for learning or modeling based on people’s preferences, but its outstanding behavior was still achieved.
It proves The model was developed by Meta in collaboration with Carnegie Mellon University, University of Southern California and Tel Aviv University. According to proven researchers, LIMA does fantastically well and learns to follow certain patterns of responses with a few examples given in its training. It is also generally good for new tasks that did not appear in the training data set.
as good or better than GPT-4 and Bard. In a controlled study by these researchers, the LIMA responses proved equivalent or superior to those produced in the GPT-4 in 43% of cases. Things got better when compared to Bard (58%) and went even further when compared to DaVinci0003 (from OpenAI) with 65%. All of this “suggests that almost all of the science of large-scale linguistic models is learned in instruction, and that only a limited amount of data is needed to teach high-performance models,” said the authors of the study. .
In the RLHF it may not be that big of a deal. One of the most important conclusions of the study is that the use of learning support from Human Vision (RLHF) technology does not bring about as many improvements as previously believed. In this system, a series of human users are rewarded with a model to optimize their behavior while exercising. It is a valuable process that they use in OpenAI to develop their models and, for example, they use GPT-4 to make the model better.
The grace of day and night hypothesis. According to Meta, this raises a hypothesis in which the night time, as they say, after the first training should be aimed at teaching the model a certain form or style to be able to use in interactions with users. So this model “tuning” is more about style than substance (more about quality than quantity, you might say).
But and Thus, the LIMA research team highlights that building these data highlights with high-quality models is quite a challenge and is not always a scalable option. Even with these results, LIMA is still slightly below GPT-4: it generates good answers, but a special prompt that leads it in trouble or in a bad example so that it can introduce not so accurate answers.
LLMs have something to lose. For Yann LeCun, from Meta, the behavior of LIMA shows that investing in the development of new and important LLMs is important in the short term, but it will not be in the medium term, “at least not without some big changes”. he said. in a recent tweet.
In Xataka | Meta would lose the AI class: it just made a 180-degree turn with its special subscription message