Article By : Don Scansen
There is no reason good reason that imaging sensors should always emulate human vision…
Day two of ISSCC 2021 provided a second pair of plenary talks both providing ample thought nourishment. Let’s start with one from a veteran of solid-state imaging, who reminded the industry that there is no reason good reason that imaging sensors should always emulate human vision.
“More to the picture than meets the eye”
Thanks to that line from Neil Young, another entertaining and insightful talk was given by world-renowned imaging technologist Albert Theuwissen. Those who have been lucky enough to have a chat with the affable Doctor Theuwissen will have probably heard about his passion for the Canadian rocker. The title borrowed from the song “Hey Hey, My My” is duly credited in the ISSCC 2021 digest.
Outside the realm of the ISSCC, there was another recent connection between the entertainment world and the technology of CMOS image sensors. A second big name in imaging technology recently received an Emmy Award. The National Academy of Television Arts and Sciences (NATAS) for “inventing and pioneering the intra-pixel charge transfer CMOS image sensor.” (That’s at least one Emmy that I can support.)
CMOS image sensors are ubiquitous and very often underappreciated. With our modern dependence on smartphones (remember when we called them “cameraphones”?), people are capturing images constantly. A nice photo of that cuddly cat or smiling grandchild might provide some appreciation for the convenience, but it’s unlikely to foster much thinking about the tech behind the pic.
Theuwissen explained the technology and the unique challenges faced throughout the history of solid-state imaging. Designing and building a CMOS image sensor requires the collaboration and engineering design excellence of a diverse set of experts (optical, materials, manufacturing, analog design, mixed signal design, and digital design) all for a single monolithic image sensor.
The early goals for solid-state imagers related to catching up to the human eye, but imager technologists now look to applications well beyond human capability. The ability of silicon to “see” beyond the visible opens up the possibility for imagers that are not blinded by rain, snow, or fog. Go a little further into the infrared, and blood is transparent, suggesting new medical diagnostics.
But the most promising future application is the integration of CMOS image sensors with higher powered processing. Theuwissen suggested that perhaps the most promising new products will be imager sensors stacked with neural processing chips.
Sony was a pioneer stacking the image signal processor die with its detectors, a technology that has become a mainstay in smartphone cameras. Sony has another paper at this conference related to this idea, so the future might be upon us. In paper 9.6, Sony will present a 12.3 megapixel back illuminated sensor stacked on a convolutional neural network processing IC.
Anyone looking for a summary of the rich history of the technology with all the references of the major milestones, this is exactly the review paper for you.
But this talk ended as something of a cautionary tale. With an eye toward ever more advanced information processing integrated with CMOS image sensors, there are privacy concerns as most urban and suburban areas are blanketed by a vast network of cameras. We are nearly constantly under the watchful eye of someone or something.
To put icing on a great technology review talk, Theuwissen ended with a request for ISSCC organizers to consider a push toward the concept of “responsible innovation.”
That was my key takeaway from this presentation.
The other plenary slated on day two brings to mind the necessity of just how important responsible innovation is.
Radios are everywhere too
Dina Katabi from MIT presented some very nifty health monitoring ideas using radio frequency waves. The MIT professor noted with the proliferation of radios and even sensor nodes with wireless capability that the radio portion is only used to communicate data, but the possibility exists to use the radio energy directly for sensing.
X-ray Vision
You might think the idea of “seeing” through walls is the stuff of science fiction, but not for the team from MIT. Katabi and her students have found ways to exploit non-linearities in the interaction of RF energy with the human body to effectively map a range of activities, function, and vital signs with radio waves while transmitting and collecting the backscattered energy through walls.
The systems shown in the demonstration videos were quite impressive. One of the most promising was the non-contact system for sleep monitoring. Sleep labs are common these days for diagnosing sleep apnea among other disorders. But these tests require subjects to be hooked up to a full wiring harness and furthermore take place in the strange surroundings of a lab. I’m sure most of us would exhibit some sort of sleep disorder under those circumstances. The cost and complexity tend to make this a one-shot experiment for the diagnosis.
With remote monitoring with radio frequency waves, Katabi’s group achieved very close to sleep lab capability while subjects can sleep in their own beds. Furthermore, data can be easily collected over an extended time period compared to sleep monitoring lab facilities.
Other experimental results proved the capabilities of tracking the movement and posture of human subjects and detecting respiration. There are certainly some applications with elder care. It’s hard to escape a discussion of Covid-19 even at a solid-state circuits conference. Of course, we know why it’s virtual, but the Katabi group had a more direct connection.
With patients recovering at home, there is a need to remain isolated. Unfortunately, there is also the possibility for a relapse. The remote radiofrequency monitoring of recovering Covid-19 patients provided monitoring of improving health and early warning signs of unlucky souls who required a return visit to the hospital.
With the capabilities of this technology, even more so than imaging, I think “responsible innovation” needs to be front of mind.
