“The COVID-19 outbreak has dramatically changed the way people work, and the laptop market has boomed as a result. Long before the virus spread globally, industry analyst firm Gartner noted in January 2020: “Strong business demand for Windows 10 upgrades has seen the PC market grow for the first time since 2011, particularly in the U.S., EMEA ( EMEA) and the Japanese market.” To remain competitive in this growing market, notebook OEMs want to differentiate their products by adopting new technologies and adding new applications to increase productivity.Although its
“
The COVID-19 outbreak has dramatically changed the way people work, and the laptop market has boomed as a result. Long before the virus spread globally, industry analyst firm Gartner noted in January 2020: “Strong business demand for Windows 10 upgrades has seen the PC market grow for the first time since 2011, particularly in the U.S., EMEA ( EMEA) and the Japanese market.” To remain competitive in this growing market, notebook OEMs want to differentiate their products by adopting new technologies and adding new applications to increase productivity. While many of these technologies were originally used in devices such as smartphones, they can also shine on laptops, offering the power of full-fledged laptops on the one hand and the convenience and security of mobile devices on the other. and adapts well to the surrounding environment.
Many of the new features on laptops can be implemented through software, but this reduces battery life. In order to solve this contradiction, Lattice semiconductor is developing a hardware-based solution to achieve higher performance with extremely low power consumption, and at the same time, due to the use of Lattice FPGA devices, it provides great convenience for future upgrades. Because FPGAs can be reprogrammed in the field, OEMs can also download software and firmware updates to implement new applications on the hardware without redesigning the hardware.
Laptop use continues to grow
It has been predicted that laptops will be replaced by smartphones and tablets, however due to changes in the work environment and the development of emerging technologies, the use of laptops has increased rather than decreased. Cloud computing and ubiquitous broadband connections mean that employees can be productive outside the workplace as well. With full-size keyboards, high-resolution displays and cameras, laptops are often the first choice for business productivity applications such as email, word processing, and video conferencing.
While many of the technological innovations around power saving and artificial intelligence (AI) are mostly applied to smartphone platforms, they can also be applied to laptops without compromising their power consumption. For example, the Display screen of a laptop is the main source of power consumption. We can intelligently monitor the user to determine whether the user’s attention is on the computer screen. When the user’s attention is shifted to other places, the screen brightness can be automatically reduced, thereby greatly reducing the Extend battery life. The app is already widely available on smartphones, and a similar experience is now available on laptops.
The growing popularity of AI
Although AI is still an emerging field with unlimited potential, it is continuing to demonstrate its enormous value to consumers. Some AI applications have become mainstream, and users are very satisfied with the convenience they bring. Facial recognition is one of them, especially as a security feature on smartphones. It is also now widely used at airports around the world to track travelers entering or leaving the country as they pass through immigration. In each of these cases, the camera recognizes faces, increasing the confidence of a computer user (or a tourist entering a country) in their determination of their identity. Today, consumers are increasingly comfortable with the AI experience, and they expect smart vision features like facial recognition to be more widely available on laptops.
Security and Privacy
Security and privacy are important for desktops, but even more so for laptops. Whether for work or personal use, security and privacy (though technically distinct concepts) can be considered a requirement, as the need for security performance to protect sensitive data goes hand in hand with the need to keep that data private.
We all know that protecting security and privacy often means sacrificing convenience. While those burdensome security measures can protect laptops used in public areas, in familiar, controlled environments, such as the workplace or home, they can be annoying or even unnecessary.
The laptop can use AI to recognize the scene, and the benefit is that it can adjust security settings based on its location. When using it at home, the settings can be adjusted to make it easier to use due to less objective risk. In contrast, when used in a coffee shop, the security settings can be automatically increased due to the unfamiliar environment and people.
Of course, it’s more than just detecting the environment the laptop is in. Cameras that scan the environment can also detect when the user gets up and leaves, such as to pour coffee or go to the bathroom. The computer can then quickly lock itself without requiring the user to lock the screen. Likewise, the camera can detect that someone else is peering at the user’s screen from behind, so it can take steps to prevent unauthorized users from viewing the screen. Computer intrusion events are emerging in an endless stream today, and AI can also help monitor internal computer behavior and activities, reporting any suspicious events to users.
The above list is a few examples of laptop upgrades designed to meet the needs and expectations of the modern worker. Due to limited space, this article only covers two laptop trends:
• Real-time online and instant-on features. The main purpose is to save power.
• A new laptop form factor. That means detaching the screen from the body, experimenting with entirely new laptop shapes, or folding the laptop and screen.
Lattice FPGAs, specifically the iCE40 UltraPlusTM and CrossLink-NXTM FPGAs, and the collection of Lattice sensAITM solutions for real-time online network edge AI applications provide an efficient way to implement these capabilities in the laptops of the future. Moreover, its power consumption is much lower than ASIC-based solutions.
Live online and instant start
Today, there are only three ways a laptop can go to sleep or wake up: press a button (perhaps with a mouse), close the lid, or wait for the computer to go to sleep. Newer laptops may wake up with a voice command.
The main problem facing laptops is their SoC chips. The SoC is responsible for controlling the various activities that take place in the laptop, so it is involved in the vast majority of tasks performed by the computer. An SoC contains the CPU core and possibly other processing circuits and peripheral circuits, and it consumes a lot of energy while it is working.
Since the SoC is involved in a lot of the laptop’s activities, the computer can hardly handle any tasks while the system (especially the SoC) is sleeping. The functions available during sleep are the options related to waking up the system. No other tasks can run on a sleeping system. In some cases, if you need to handle specific tasks (such as downloading email, network monitoring, etc.), the operating power can reach 100 milliwatts, which greatly reduces battery life.
If a laptop has AI capabilities that respond to sensor signals, the computer can perform many tasks even when the SoC is sleeping. AI functions can work or make decisions while the SoC sleeps, either waking the SoC for further work, or keeping the SoC asleep and letting the AI handle the tasks it needs to accomplish. Low-power FPGAs are ideal hardware platforms for implementing AI functions. They have enough processing power to perform basic AI tasks (such as presence detection) while consuming much less power than SoCs. If the AI determines that the SoC is required to participate in the task, it will wake up, cooperate with the AI to process, or take other measures.
The Lattice sensAI solution collection can be equipped with iCE40 UltraPlus, ECP5™ and CrossLink-NX FPGA to easily implement the above AI functions. But so far, such AI has mostly been applied to Internet of Things (IoT) devices. For example, a smart doorbell might have the following features:
• Detect the presence of people and other objects such as packages.
• Detect and/or recognize a specific face or person. This can be a specific person registered as an authorized user (such as the occupant of a house), or someone who frequently passes by the doorbell camera, such as a uniformed courier or gardener.
• Understand some gestures or voice commands. Combining facial recognition with voice keyword or password verification enables two-factor authentication login for increased security.
• Minimize data transfer to the cloud with an inference engine compatible with a wide range of sensors.
In such applications, Lattice FPGAs can provide higher performance than stand-alone CPUs at lower power consumption when the SoC is inactive, because CPUs must perform tasks through software, while FPGAs use more efficient, based on hardware implementation. For example, a CPU controlling a smart doorbell or surveillance camera can process 1-2 frames per second while consuming up to 100 mW. Lattice iCE40 UltraPlus FPGA can process 5 frames per second at 7 mW power consumption, which is 3-5 times the performance of the former, and the power consumption is only one-fourth of the former.
If Lattice’s sensAI solution can be used in a smart doorbell or surveillance camera, it can also be used on a laptop, enabling the following functions:
• Detect when the user is watching the screen. Dim the screen to save power when the user’s eyes are off the screen; increase the screen brightness when the user’s attention refocuses on the screen.
• Adjust security settings based on the location of the laptop.
• Detect the user’s gaze direction. This can simplify tasks that usually require a mouse. Eye tracking can avoid the need to manually perform certain frequently occurring tasks.
• Optimized power consumption for graphics rendering. You can focus on rendering the portion of the screen that the user is watching. The rest remains relatively blurry or rendered at a low level, saving power until the user’s gaze shifts to other parts of the screen.
• Affect the narrative trajectory of the game through the player’s eye movements.
• Detect if someone is peeking behind the user.
• When the laptop is not in use, dim the screen to save power.
The schematic below illustrates how the Lattice iCE40 UltraPlus FPGA offloads some of the AI processing from the laptop SoC.
In this design, the sensor interface collects the inputs from the individual sensors, processes it in the FPGA’s neural network (NN) accelerator, and stores the weights and activations needed to process the neural network in on-chip SRAM. If further processing is required, the accelerator makes a decision to wake up the SoC.
The AI capabilities were built using various resources from the Lattice sensAI solution collection as well as iCE40 UltraPlus and CrossLink-NX FPGAs. Designers can further tune networks trained on major frameworks such as Caffe, Keras, MobileNet, ResNet, SSD, and TensorFlow using sophisticated software tools and a complete set of design tools. sensAI includes reference designs, demo boards, IP cores, and hardware platforms for developing a variety of common AI applications (eg presence detection, object counting, facial recognition, etc.). Lattice also offers custom design services that allow developers to quickly implement AI capabilities without having to fully learn how to develop and implement AI applications.
Reduce the number of signals and enter a new form
The second challenge for modern laptops is how to go beyond the traditional look. This involves the flexibility of the notebook, such as folding the screen in the middle to stow it away. Laptop designs face the challenge of transmitting a large number of signals, which in turn involve numerous protocols.
A simple approach is to reduce the number of signals through signal aggregation. Multiple low-bandwidth interfaces can be multiplexed onto a single higher-bandwidth signal and then demultiplexed, greatly optimizing laptop cabling. It makes connections between boards simpler, it also simplifies wiring on the board, and it means that wiring through hinges or other flexible components can be greatly reduced. And because the number of connectors is reduced, reliability is also enhanced.
Signal aggregation can be implemented on a single wire or a pair of LVDS. Lattice’s FPGAs offer many performance options for transporting these signals, allowing for the most efficient design implementation. For example, an iCE40 UltraPlus FPGA can achieve a transfer rate of 7 Mbps on a single wire. With higher performance Lattice FPGAs, higher bandwidth aggregation speeds can be achieved.
The problem, however, is that different signals may be transmitted using different protocols. This dilemma can be solved with an I/O hub similar to a sensor hub (Hub). The hub aggregates different types of local I/O signals and multiplexes them onto a smaller number of high-speed lines. These lines connect to the I/O hub on the other end, where the signals are restored to their original protocol type.
Lattice FPGAs provide the physical hardware for receiving I/O signals and transmitting multiplexed signals. Lattice also provides IP for building I/O hubs. Signal aggregation can be applied to all types of sensors, so data streams of different types of sensors (RGB, IR, LiDAR, radar, etc.) or signals transmitted over different bus types (I2C, I2S, UART, GPIO) can be aggregated on the same pin or on the line. Lattice’s long-standing experience in signal aggregation for smartphones is equally applicable to sensor aggregation applications on laptops.
Lattice helps you build the laptop of the future
Engineers are working hard to develop new laptops that provide users with a smartphone- and tablet-like experience. Tomorrow’s laptops will be easier to use, have longer battery life and be more secure than today’s products.
The improvements in power consumption and form factor will be most noticeable. Laptops will be powered by power-saving technologies that allow users to use them continuously for a whole day or more. They will also become more flexible and portable.
And Lattice FPGA will be the key to realizing these new functions in notebook computers. Processing work off the SoC extends the SoC’s sleep state, waking it up only when it needs to use its computing power, which can significantly reduce power consumption. Implementing AI in hardware, as well as a sensor hub that can connect multiple cameras, microphones and other sensors, means the SoC is down for longer. I/O hubs that aggregate multiple signals can help reduce the number of signals and enable AI-based sensor aggregation, making laptop building simpler and providing a richer user experience.
Lattice’s FPGAs provide the hardware platform for these hubs, and the sensAI solution collection makes AI capabilities more accessible. The combination of the two could help developers build lighter, lower-power laptops that provide more powerful productivity tools for business and personal users in the 2020s.
