Grants and Contributions:

Grant or Award spanning more than one fiscal year. (2017-2018 to 2022-2023)

The Human Visual System is supported by a very high resolution fovea with rapidly declining resolution in the periphery. The fovea captures details in about a 2 degree cone extending from the center of the eyes. Thus, we can only see a few letters of text clearly at a time at reading distances. In our mind, however, we think that everything is clearly visible. This perception is a result of our eyes being dynamic or “active” and always being guided by the brain to look at precisely what is most important at a given instant.

In my basic research I introduced the concept of foveation for image, video and 3D compression. Furthermore, considering eye movements I developed the first active calibration of cameras without using any known patterns or by matching individual feature points. It is the first calibration approach that is consistent with the pan, tilt and torsional rotations of the human eye, answering some deeper questions on human vision. I also considered supporting the wide field of view of the human eyes and introduced “Panoramic Stereo” using one camera. These fundamental research topics have impacted the way coding standards have incorporated “region of interest,” and have resulted in the creation of spin-offs, like PVSI and VisionSplend, over the years by collaborators and trainees.

Designing multimedia systems following biological motivation is only the first part of my approach to addressing several problems, the second part complementing this are the algorithms and their analyses. For the second part my major emphasis is on probabilistic approaches. For example, by statistical analysis of the distribution of errors I proved why my active algorithms are much more robust. Through an average case analysis I demonstrated the efficiency of our Lagrangian advection. Through probabilistic random walks I improved image fusion. I used stochastic perturbation for robust matching and registration. Finally, I also introduced the use of stochastic processes for reliable detection of brain injuries in prematurely born infants.

Over the next five years I plan to introduce the following novel components: (a) incorporating foveation into panoramic stereo; (b) making further improvements to Motion Capture data compression by studying the role of attention of viewers; (c) combining saliency (or the detection of important regions in an image) with foveation for better multimedia (image, video and 3D) coding with respect to human observers; (d) making active camera calibration more robust; and (e) developing new approaches in medical and surgical image analysis. The development and analysis of my algorithms will be grounded in thorough probabilistic techniques and analysis, as in my past research.

The proposed research, if successful, will have significant impact in next generation panoramic, 3D and plenoptic multimedia capture, processing, and transmission, as well as in medical and surgical innovations.