Centering around chips, the Snapdragon sparks a revolution in edge AI.

Redefining the use of chips, the Snapdragon ignites a groundbreaking movement in edge AI.

【Mobile China】In the history of the development of smart terminals, chips have always played a crucial role. The reason why chips have become an important driving force for the development of smart terminals is due to the productivity improvement brought about by each upgrade in chip performance. As we enter 2023, AI models have exploded with unimaginable application effects, leading to a noticeable emergence of intelligence. With AI technology fueling the fourth industrial revolution and bringing about a new era of productivity, it has become a universally discussed topic worldwide. In such times, the trend towards AI-powered smart terminals seems unstoppable. In order to adapt smart terminal AI development to the rise in demand for intelligent productivity, we urgently need a more powerful “core”.

Against this backdrop, Qualcomm has released two new products at this year’s Snapdragon Summit – the third-generation Snapdragon 8 and Snapdragon X Elite. These products, respectively catering to smartphones and PCs, enable local operation of large-scale models with billions of parameters, endowing mobile devices with greater intelligence.

In recent times, the upgrade and iteration mindset for mobile chips has been very clear, with steady improvements in process technology allowing chips to constantly expand in terms of performance. So, chips come first, followed by consumers using new electronic products. Chips are the starting point of the smart device development chain, defining the performance of products and anchoring the direction of industry development.

However, with the rapid development of technologies such as the Internet of Things, 5G, and artificial intelligence, the scope of smart device demands for chips has expanded. For example, artificial intelligence has introduced more diverse chip requirements. Supporting artificial intelligence GPUs, NPUs, low-power and low-cost communication chips, functional chips, and more have become the new starting point for the development chain of the chip industry, signifying a brand new stage of development in the mobile chip field.

• Redmi K70 Series Leads the Way Who is the Strongest Snapdragon 8 Gen2 in the 3000 Yuan Range?
• 11 models of sweeping robots fail the spot check, but Stone P10 and T8 Plus have successfully passed the national standard inspection.
• Redmi K70E key specifications exposed, equipped with Dimensity 8300 or to be released soon.

In this new stage, Qualcomm has been forward-looking in its strategic vision, early on considering AI as a key capability in chip development. As early as 2015, Qualcomm had already introduced the Qualcomm AI Engine, which supports heterogeneous computing, on its mobile platforms. This engine can mobilize the entire chip’s capabilities to enhance image, audio, and sensor processing through AI. Additionally, in 2016, Qualcomm released the Snapdragon Neural Processing Engine SDK, which allows OEM manufacturers to run their own neural network models on Snapdragon 820 devices such as smartphones, security cameras, cars, and drones. This allows downstream manufacturers to customize development based on the AI capabilities of Snapdragon chips, such as enhancing image, audio, and sensor processing with AI.

After years of technological accumulation, starting from the first generation Snapdragon 8, Qualcomm has increasingly focused on the AI computing power of its chips. It was the first to support INT8 integer calculation, achieving a doubling of the tensor accelerator’s computation performance. With the second generation Snapdragon 8, the overall AI performance has increased by 4.35 times. The continuous improvement in AI capabilities has helped downstream manufacturers expand their terminal development scenarios and significantly increase the operating speed of terminal products. This has led to a significant reduction in the reaction and computation time required to perform the same operations, making AI the most crucial foundation capability for chips today.

For Qualcomm, continuously improving chip performance is not the ultimate goal of innovation. The true goal is to use chips as a foundation to enhance the production efficiency of smart devices, transform the way users use devices, and improve the user experience. Therefore, to Qualcomm, the role of AI is irreplaceable in this new phase of comprehensive improvement in mobile terminal experience. This is also true in the eyes of consumers. In the past, chip evaluations often emphasized performance, such as frequency, benchmark scores, and frame rates. However, ordinary consumers now attach increasing importance to the AI capabilities of chips, and AI capabilities are becoming important content of consumer attention.

Therefore, fully utilizing technological advantages and achieving the evolution of AI capabilities in chips have become the main direction of Qualcomm’s chip upgrades. Earlier this year, Qualcomm demonstrated its powerful terminal-side AI capabilities by running Stable Diffusion, an AI model with over 1 billion parameters, on a mobile phone. The image generation speed was controlled within 15 seconds.

Now in October, terminals equipped with the third-generation Snapdragon 8 processor can not only run billion-parameter large models locally but also generate images using Stable Diffusion in just 0.6 seconds. Such a huge improvement is inseparable from the core of Qualcomm’s AI engine, which is the upgraded Hexagon NPU with a new microarchitecture. Its performance has improved by 98% and energy efficiency by 40%. In addition, the Hexagon NPU also integrates a hardware acceleration unit, micro-slice inference unit, performance-enhanced tensor/scalar/vector accelerator units, and all units share 2 times the bandwidth of large-capacity shared memory.

This means that users can freely use AI capabilities to achieve generative AI experiences, such as generating images and text, on terminals equipped with the third-generation Snapdragon 8. This enhances users’ creativity and productivity on terminal devices. Chris Patrick, Senior Vice President and General Manager of Qualcomm’s Mobile Business, said, “The third-generation Snapdragon 8 platform will usher in a new era of generative AI, empowering users to create unique content, enhance productivity, and achieve other breakthrough use cases.”

Outside of the mobile chip battleground, Snapdragon also has high hopes for the PC market. At this Snapdragon summit, Qualcomm also released the Snapdragon X Elite for PC products, which is also a key step in Qualcomm bringing AI to the edge.

The Snapdragon X Elite is based on the custom Oryon CPU core, and under the same power consumption, its CPU performance can reach twice that of competing x86 processors. The peak multi-threaded CPU performance is 50% higher than that of the Arm-based Apple M2 chip. In terms of GPU, the Snapdragon X Elite outperforms x86 integrated GPUs by as much as 80%.

In terms of the most important overall Qualcomm AI engine AI computing power, it has reached 75 TOPS, an increase of approximately 100 times compared to 2017. Not only is there an upgrade in computing power, but the Snapdragon X Elite also integrates Qualcomm’s central sensor, MicroNPU, and always-on ISP. It is worth mentioning that the Snapdragon X Elite is also the first PC processor that integrates an always-on ISP.

The powerful computing power allows for generative AI experiences that were previously only available in the cloud to run successfully in the local environment of PCs with Snapdragon X Elite chips. According to reports, the Snapdragon X Elite can run large language models with up to 13 billion parameters on the PC side, with token generation speeds of up to 30 per second. In addition, Qualcomm also stated that in the future, on the PC side, edge AI will directly improve users’ specific usage experiences locally, even without network connectivity, AI will always be with the user.

While expanding the evolution of chip AI capabilities, breaking down the barriers of processors to move towards more intelligent terminals, Snapdragon is opening up new frontiers for chip innovation.

It is not difficult to see that the chip industry has reached a crossroads, where having more powerful performance is just the basic requirement, and having strong AI capabilities is what can bear the responsibility of leading the next era. In Qualcomm’s eyes, edge AI is the main direction of future development.

Because compared to traditional AI services that rely on the cloud, edge AI has obvious advantages such as high independence, fast response speed, and security and reliability.

First, from the perspective of response speed, in the process of using cloud-based generative AI, there is a high latency during peak demand periods, while edge AI will not have such a problem. On terminals equipped with the latest Snapdragon chips, Stable Diffusion generates images in less than a second. In the case of poor network conditions, the AI capabilities of local devices become the user’s only choice, which is the primary advantage of edge AI.

Secondly, from a security perspective, terminal AI data is stored and processed on the user’s own device, avoiding the risk of information leakage. In addition, with secure chips and encryption algorithms, it can more effectively protect user privacy. This is also an important reason why it is superior to cloud-based services.

On the other hand, AI can provide highly personalized intelligent services by learning users’ long-term accumulated personal behavior patterns and preferences on their local devices. It understands users’ language habits and contextual environment, saving them a lot of time and effort. This is an inevitable requirement for realizing the true essence of an “assistant”.

Finally, according to Qualcomm’s CEO Amón, “The predictive ability of AI, combined with terminals designed specifically for you, enables intelligence in terminals, operating systems, applications, and the cloud.” It can be foreseen that as Snapdragon’s AI capabilities evolve, the AI capabilities of Snapdragon will quickly expand from PCs and smartphones to more intelligent terminals. This means that the curtain of the fourth industrial revolution, in which intelligent devices are powered by AI, is slowly opening, and a new era shaped by Snapdragon is also coming.

We will continue to update Phone&Auto; if you have any questions or suggestions, please contact us!