Cognitive scientists, geologists, and imaging scientists are teaming up to learn how expert geologists unconsciously view landscapes for clues that point the way to important discoveries. The National Science Foundation has awarded the interdisciplinary team, led by Robert Jacobs of the University of Rochester, $2 million over the next five years to find the answers. In addition to Jacobs, the team includes John Tarduno from the University of Rochester and Jeff Pelz, Jinwei Gu, Nate Cahill and Karen Edwards from the Rochester Institute of Technology.

Not only might the findings shed light on how a seasoned geologist's brain teases out information from a terrain full of complex features, but the results might be applicable to scientists in other fields. Crucially, the findings could help reduce the costs of training young scientists by giving them useful simulated field training without the high costs of travel and equipment.

"This is a new way to look at how experience changes how we see," says Robert Jacobs, professor of brain and cognitive sciences at the University of Rochester, and lead investigator of the project. "In the past, there had been some attempts to understand how radiologists read an X-ray image, but no one has ever done something like this—where we monitor people out in the natural environment and try to understand how experienced and inexperienced scientists see the same scene differently."

John Tarduno, pro

fessor of earth and environmental sciences at the University of Rochester, led 10 undergraduate students on a nine-day excursion, March 7–15, 2010 connected to the introductory geology course, Earthquakes, Volcanoes and Mountain Ranges in California: A Field Quest. The students visited various areas of geologic interest, such as the San Andreas Fault, the snowy Sierra Nevada near Yosemite National Park, and the deserts of eastern California including Death Valley.

Jeff Pelz, professor of imaging science at the Rochester Institute of Technology,

and Jason Babcock tracked the eye movements of the UR students and three experts observing the geological formations. The subjects wore an eyetracker developed by Babcock’s company, Positive Science, containing two DVRs, and donned special glasses equipped with two tiny cameras—one pointed at the their right eye, the other at the observed scene. Each subject also wore a wide-brimmed hat to block sunlight from the cameras. The first year of the project has already provided the largest collection of gaze data in a group of individuals outdoors.

Back in Rochester, the team will analyze the data, noting the way each student takes in the geological formations around them, and comparing that to the way experts viewed the same formations. "An expert geologist scanning a landscape might notice where a stream is offset from its normal course and deduce that motion on a fault had caused the stream diversion," says Tarduno. "Streams and patterns of erosion in general provide clues to processes shaping landscapes. But we want to know more about how experts make the connections visually. The data could be invaluable in letting us know how a student sees the landscape and how we can teach them to see it through more experienced eyes."

Mitchell Rosen, former research professor at the Rochester Institute of Technology, and imaging science doctoral student Brandon May collected another two terabytes worth of visual and aural data. They captured the scenery through high-resolution panoramic stills, hemispherical video and still photography and are currently preparing the images for presentation in semi-surround

environments. “Students will visit these presentations so the scientists can experiment with methods for recreating the field experience in a ‘virtual field trip’,” Rosen says. “Eye tracking and other analyses will reveal the extent to which the benefits of on-site geological training can be brought into the classroom.”

“Much of the captured imagery is really breathtaking,” Rosen continues. “Our goal was to bring back vistas that include the visual cues that geologists use out in the field to interpret geological history. With the latest high-resolution imaging combined with the mobile and computing capacity, even battery capacity—everything has come together to allow us to go out and capture these fabulous scenes from remote geological sites and present them back here as if you were actually there yourself.”

From reports by:  Jonathan Sherwood jonathan.sherwood@rochester.edu 585.273.4726  and Susan Gawlowicz smguns@rit.edu 585.475.5061