Virtual Visual Cortex

Author: Rodolfo Antonio Salido Benítez

Class: History of Art and Technology

Completion Date: March 17, 2015 (Revised November 2018)

Software: Photoshop, Matlab, Illustrator.

Style: Diagram

Techniques: Complex image processing and graphic design.

Material: Input photograph. Original file

My project consists of a visual representation of the information integration process of visual input in humans as modeled by computers. I presented this process as a multi staged process that intends to extract data in specific modalities from the input image. I originally intended to create a diagram explaining some of the biological integration mechanisms behind our vision. However, after many hours of research, I found that understanding of the biological information integration processes is very limited. I searched for methods to develop visual representations of the different known visual integration pathways with very limited success. Thus, I chose only 2 of the pathways that I thought could generate an appealing visual representation: the Gabor filter model for Cortical Simple Cells visual information integration, which effectively explains center-surround antagonism in Retinal Ganglion Cells and Orientation Tuning in Simple Cells, and a simplified color discrepancy model, which represents the sensitivity of different cones in our retina to different wavelengths of light – colors-.

The visual representations were generated by different computational algorithms in MatLab. The input image was first Gabor filtered under parameters that selected for an effective edge identification (an absolute filtering parameter was chosen to extract edges relative to the full resolution of the image) and different edge orientations (0º, 30º, 60º, 90º, 120º, 150º). This process closely resembles biological orientation tuning mechanisms of simple cells in the visual cortex of cats as demonstrated by David Hubel and Torsten Wiesel. The output images [(A) in the diagram] were then filtered by non-maxima suppression and added on top of each other to generate an edge detection image. The simplified color perception visualization was generated by softening the original image with Gaussian blur. The purpose of this was to decrease contrast in the image generated by fluctuating light intensity as these image characteristics are not part of color perception. The blurred image was then colored filtered through 3 modalities: green, red, and blue. These colors correspond to the different peak absorption wavelengths of opsins expressed in retinal cones, our color receptors. This model is a simplified color perception model because it ignores color opponency mechanisms, which detect color contrast when bordering colors have the same light intensity. The two images in the color perception visualization [(B) in the diagram] correspond to a gray scale visualization of color content (Red, Green, and Blue) within the image and a color visualization of the 3 different color channels. The final image [(C) in the diagram] is a visualization of the computational model of human visual perception used in this project. It is a result of an addition of the six different orientation channels and the 3 different color channels.

(A) Gabor filtered image at edge orientations of 0º, 30º, 60º, 90º, 120º, and 150º representing orientation tuning of simple cells in the visual cortex of cats. (B) Gaussian blurred image filtered through Red, Green, and Blue color channels representing peak absorption wavelengths of opsins in retinal cones. (C) Composite image resulting from addition of all preceding filtered images.

The design of the diagram intended to make good use of space. However, after further analyzing it, I realized that its hierarchical order doesn’t resemble human visual integration because orientation tuning involves higher order neurons while the simplified color perception pathways involve lower order receptors and neurons. I wanted it to resemble a textbook diagram because my initial intention was to provide an effective visual aid for the understanding of Orientation Tuning as I was unsatisfied by the diagram used in my neuroscience textbook. The complexity of my idealized diagram was far outside my technical reach, yet I’m very pleased with the diagram of Orientation Tuning I developed. I opted for a diagram design with depth in an effort to effectively portray a stack of several different visual integration steps.

My original intention for this project changed drastically. At first, I wanted to reconstruct an image that closely resembled reality based of biologically inspired computational analysis of image features. I wanted to piece together a photo from different discrete characteristics mathematically extracted from an input image. However, after days of research, I encountered various limitations. Thus, I shifted my focus from trying to emulate human vision to trying to bring attention to computational vision. Through this new project I wanted to bring attention to the following issues:

Computers have very limited image processing capabilities in comparison to humans, yet advances in computational models for visual integration actively fuel breakthroughs in neuroscience.
The limited image processing capability of computers is only one example of many different tasks in which humans outcompete computers. This highlights the different capabilities of humans and computers and should raise thought concerning the cooperation of humans and computers in ordinary life.

The main purpose of the final image (C) in my diagram was to provide an idea of how computers can process visual information. This image shows how machines process the world that surrounds them differently than we do. Object, site, and face recognition processes differ greatly from humans to computers. This demonstrates how differently we integrate and communicate with our environment in comparison to machines.

Finally, my goal of creating a better visual teaching aid demonstrates how heavily we rely on visual input not only to understand our physical surrounding but even to try and comprehend abstract ideas. This realization reinforces the idea of “Visualism” brought forth by Johannes Fabian.