Join our daily and weekly newspapers for exclusive content on the latest updates and industry-composure AI coverage. learn more
Outside the Massachusetts Institute of Technology (MIT), Boston -based Foundation Model Startup Liquid AI, OpenaiI’s GPT series and Google’s Gemini family are trying to transfer transformer architecture from their dependence on the transformer architecture that undersign the most popular big language model (LLM).
Yesterday, the company announced “Hina Age, International Conference on Learning Representation (ICLR) 2025.
The conference is taking place in Vienna, Austria this year, one of the major programs for machine learning research.
New condensed models promise rapidly, more memory-skilled AI shore
The higha is an engineer to improve the strong transformer baseline on both computational efficiency and language model quality.
In real-world tests on the Samsung Galaxy S24 Ultra smartphone, the model gave low delays, small memory footprints and better benchmark results compared to parameter-milan transformer ++ models.
A new architecture for a new era of Edge AI
Unlike most of the small models designed for mobile certification-Smollm2, Phi model, and LLAma 3.2 1B-hyena age are away from traditional attention-intensive designs. Instead, it strategically replaces two-thirds of equalized-converted meditation (GQA) operators with a gate conversion from the Hina-Y family.
The new architecture is the result of the synthesis of the Architecture (Star) framework to suit liquid AI, which uses evolutionary algorithms to automatically design the model backbone and was declared back in December 2024.
The Star operator examines a wide range of compositions, which lies in the mathematical principle of linear input different systems, such as delay, memory use and quality to optimize for many hardware-specific purposes such as memory, memory use and quality.
Direct benchmark on consumer hardware
To validate the glory of the real world of the hyena age, Liquid AI directly tested the Samsung Galaxy S24 Ultra smartphone.
The results suggest that the Hine Age achieved 30% faster prefils and decod delays compared to its transformer ++ counterpart, in which the benefits of speed increase for a long time.
Short sequence length prefil latcanie also defeated transformer baseline-a significant performance for the on-dehys applications metric.
In terms of memory, the hyena age used low RAM during constant testing sequence length estimates, placed it in position as a strong candidate for the atmosphere with tight resource barriers.
Better performance from transformer on language benchmark
The hyena age was trained on 100 billion tokens and was evaluated in the standard benchmark for small language models, including wickets, Lambada, Pika, Helswag, Vinogrande, Arch-Ezi and Arc Challenge.
On each benchmark, the hyena edge was either matched with the performance of the GQA-transformer ++ model, with a noticeable improvement in the wicket-text and the perplexity scores on the wicketkext and Lambada.
These results suggest that the efficiency of the model does not come at the cost of the quality of the profit-a common tradeoff for the edge-ordered architecture.
Hina Edge Evolution: A look at the performance and operator’s trends
For those who dive into the development process of the Hyena Edge, recently Video Walkthrough The model provides a compelling visual summary of development.
The video highlights that major performance metrics – including prefil latency, decod delay and memory consumption – improves gradual generations of architecture refinement.
It also offers a rare back view of how the internal structure of the hyena age was transferred during development. Viewers can see dynamic changes in the distribution of operator types, such as self-coordination (SA) mechanisms, various hyena variants and swiglu layers.
These changes provide insight into architectural design principles that help the model reach its current level efficiency and accuracy.
By imagining the trade-offs and the dynamics of the operator over time, the video provides valuable reference to understand the architectural successes that inherent the performance of the hine edge.
Open-source plan and a broad vision
Liquid AI said it plans to open a series of Liquid Foundation models including the hyena age in the coming months. The company aims to create an enabled and efficient general-purpose AI system that can lead to a scale from cloud datastery to individual edge devices.
The first beginning of the hyena age highlights the increasing ability to challenge transformers in practical settings for alternative architecture. Rapidly sophisticated AI workloads with mobile devices are expected to run a new base line to achieve model edge-ordered AIs such as hyena age.
Success of Hine Edge – Both raw performance in metrics and showing automated architecture design – Liquid AI develops as one of the emerging players that is to look into the AI ​​model landscape.
