The first laptops and mobile phones with AI capability built into the hardware, have reached India: a historic technology turning point for these products, some of which are being indigenously manufactured.
By Anand Parthasarathy
The emergence of Artificial Intelligence (AI) capability built into the hardware and software of personal technology devices has turned out to be the biggest technology breakthrough of the early months of 2024.
For nearly two years, AI has touched lay users of computing devices, as well as professionals, in new and dramatic ways – more so after the world-wide availability of ChatGPT, developed by a virtually unknown company, OpenAI, in November 2022. This created a whole new technology called Generative AI, which ‘learned’ the patterns and structure of input training data and then generated new data -- text, images, video --that had similar characteristics.
Suddenly, millions of ordinary users were able to delegate tiresome tasks like preparing a project report, a news story or even a chunk of school homework to freely available tools like ChatGPT, or its variants like Bing AI or Copilot from Microsoft or Bard from Google.
All these were cloud-based apps: one had to download them or use only with a live Internet connection. What if the AI capability was built into one’s device itself, with on-board neural processing units which could respond at lightning speed, and in real time, avoiding the delays inherent in trying to harness a Cloud-based, web-based tool?
That is the huge new value proposition of on-board AI and in the first weeks of January 2024. A flurry of announcements saw the PC and the smart phone emerge as the first personal devices to have AI built into them
AI built into personal computers
What is an AI-capable PC (which includes the laptop)? Technology analysts Canalys, provide a narrow initial hardware-focused definition: a PC requiring dedicated chipsets or blocks to run AI workloads on-device. They add:
“As dedicated AI blocks become mainstream in processors, the current definition will be insufficient. Additional considerations around hardware benchmarks, pre-installed models and use case performance will be needed to properly distinguish AI capabilities. This needs an adaptable approach to defining and segmenting AI-capable PCs as the market matures.”
Canalys suggests short-, medium- and long-term considerations to define the on-device AI experience: The current definition as stated above, will serve for now. But in the medium term, an AI PC is expected to be capable of 10 TOPS or Trillion Operations Per Second, using 16 GB or more of RAM memory. In the future such PCs will be capable of handling over 13 billion parameters and will take less than one second to AI-generate an image.
AI-capable PCs are poised to achieve mainstream adoption in the coming years. Under the current definition, Canalys suggests that “almost one in five PCs shipped in 2024 will qualify as AI-capable, rapidly growing to over 60% by 2027.”
Major PC manufacturers have already launched new AI-capable PCs. They will make use of processors which are integrating Neural Processing Units (NPUs) into their hardware to power this new wave of AI-capable PCs. They will also match the increasing use of Microsoft’s new Windows 11 operating system, triggered by the imminent end of support for Windows10.
A dedicated AI key
The PC keyboard has remained virtually unchanged for 30 years since a shortcut key for Windows was added. Now keyboards come with another -- artificial intelligence (AI) --key: It will allow users to access the Copilot AI tool, on new Windows 11 PCs. Copilot is based on OpenAI's GPT-4 large language model. The keyboard was previewed at the 2024 edition of CES, the Consumer Electronics Show in Las Vegas in January 2024 – an event which saw many PC makers unveil the first AI-on-board PCs.
Such PCs –for example the Gigabyte gaming laptop –allows the user to access a suite of AI-powered utilities: AI Power Gear which extends battery life; AI Boost which improves gaming performance by detecting and overlocking automatically and AI Generator, which includes various generative-AI apps for quick startup. The first AI PC to reach India in late January 2024, was the ASUS ZenBook 14 OLED with the latest Intel Core Ultra processor and a dedicated NPU.
In recent years, the PC -- whether in its desktop or laptop avatar--has seen a shrinking market in India and abroad, but AI built into the PC may change that. Privacy and security are becoming critical for businesses and consumers who are into generative AI tools. For such users, on-device AI gives peace of mind that their private data is not doing a to-and-fro with some AI cloud tool somewhere, but resides securely on their PC. Having AI built into PC hardware also means connectivity problems don’t degrade or restrict use of AI tools and AI-fuelled work can go on uninterrupted.
For these reasons, analysts are predicting that the emergence of AI-capable PCs are expected to reinvigorate the market and transform user experience in four directions: efficiency, productivity, collaboration and creativity.
2024: The Year of AI-on-Phone
Early January 2024, marked a crucial turning point in the evolution of the mobile phone – maybe the most significant since 5G or the fifth iteration of its technology which Indians experienced in 2022.
On January 17, at a global event in San Jose, in the heart of the US Silicon Valley, Samsung unveiled the Galaxy S24 handset – already being called the world’s first ‘AI Phone’.
It is the first mobile phone anywhere that has put artificial intelligence capability directly in the phone rather than accessing it from a cloud resource.
Interestingly, this handset is also being Made in India. Samsung’s South West Asia CEO J.B. Park told the Economic Times, on the day the Galaxy AI phone was launched: “These are 100% made in the Noida factory – and importantly, we are exporting these devices globally.” In fact, the Noida plant in India’s National Capital Region (NCR) is the largest smartphone manufacturing facility in the world with the capacity to roll out 120 million handsets a year.
The ‘AI Phone’ offers features like Generative AI-fuelled photo editing; more intuitive Chat Assist that can help perfect conversational tones to ensure communication sounds as it was intended: like a polite message to a coworker; Transcript Assist which uses AI and Speech-to-Text technology to transcribe, summarize and even translate recordings; gesture-driven Circle to Search with Google, where with a long press on the home button, users can circle, highlight, scribble on, or tap anything on the S24’s screen to see helpful, high-quality search results.
The phone is able to do this because – as in the case of AI PCs – manufacturers of the microchip that fuels mobile phones have provided new editions that have AI capability built into the hardware. The chip, in this the Qualcomm’s Snapdragon Gen 3, allows the phone to do things for which users hitherto used Web tools like ChatGPT.
Here is a vision of what such AI chips can do: A virtual assistant that can extract key topics from a phone call, summarize them into bullet points and then provide recommendations. A photography tool allows you to zoom on to a photo already captured and generate more details beyond the frame to make it look like it was taken on a wide-angle lens. Other photo editing tools allow you to make yourself larger or smaller, make it look like you are jumping high like a superman, or even replace the ground itself.
Not all these features will be available to start with. Analysts at Canalys suggest that there will be some on-device AI features like Live-Transcribe, but many new AI features and solutions will be rolled out in the future using software updates.
Says Anisha Bhatia, Senior Technology Analyst at Global Data, a leading data and analytics company:
“On-device AI will bring internet-less search capability to mainstream phones while keeping data more personalized and private since the data remains on the device rather than being sent to the cloud for processing. Moreover, while the benefits of 5G were not as visible to the consumer, AI’s benefits should be more apparent; the more a consumer uses AI, the more enhanced it gets with additional use cases emerging. And what better place to include AI than on devices that consumers live their lives on?”
World-wide, more than one billion smartphones with built-in generative AI are expected to be shipped by the end of 2027, according to estimates of Counterpoint Research.
Today, such AI-driven devices like desktop PCs, laptops, and smart phones appear as novelties. But if the past history of personal technology is any guide, it will take between 5 and 10 years for the AI-driven features to become so common that we, the users, will take them for granted and say about the devices we were using prior to 2024: ‘What primitive products they were’!
This has been carried in Science Reporter May 2024: SR Vol.61(05) [May 2024] pp 21-23