From its earliest inceptions, science fiction has blurred the line between reality and technological fantasy in a remarkably prescient manner. Many of the discoveries and gadgets that have integrated seamlessly into modern life were first preconceived theoretically. More recently, the technologies behind ultra-realistic visual and motion capture effects are simultaneously helping scientists as research tools on a granular level in real time. The dazzling visual effects within the time-jumping space film Interstellar included creating original code for a physics-based ultra-realistic depiction of what it would be like to orbit around and through a black hole. Astrophysics researchers soon utilized the film’s code to visualize black hole surfaces and their effects on nearby objects. Virtual reality, whose initial development was largely rooted in imbuing realism into the gaming and video industries, has advanced towards multi-purpose applications in film, technology and science. The Science Channel is augmenting traditional programming with a ‘virtual experience’ to simulate the challenges and scenarios of an astronaut’s journey into space; VR-equipped GoPro cameras are documenting remote research environments to foster scientific collaboration and share knowledge; it’s even being implemented in health care for improving training, diagnosis and treatment concepts. The ability to record high-definition film of landscapes and isolated areas with drones, which will have an enormous impact on cinematography, carries with it the simultaneous capacity to aid scientists and health workers with disaster relief, wildlife conservation and remote geomapping.

The evolution of entertainment industry technology is sophisticated, computationally powerful and increasingly cross-functional. A cohort of interdisciplinary researchers at Northwestern University is adapting computing and screen resolution developed at DreamWorks Animation Studios as a vehicle for data visualization, innovation and producing more rapid and efficient results. Their efforts, detailed below, and a collective trend towards integration of visual design in interpreting complex research, portends a collaborative future between science and entertainment.

Not long into his tenure as the lead visualization engineer at Northwestern University’s Center For Advanced Molecular Imaging (CAMI), Matt McCrory noticed a striking contrast between the quality of the aesthetic and computational toolkits used in scientific research versus the entertainment industry. “When you spend enough time in the research community, with people who are doing the research and the visualization of their own data, you start to see what an immense gap there is between what Hollywood produces (in terms of visualization) and what research produces.” McCrory, a former lighting technical director at DreamWorks Animation, where he developed technical tools for the visual effects in Shark Tale, Flushed Away and Kung Fu Panda, believes that combining expertise in cutting-edge visual design with emerging tools of molecular medicine, biochemistry and pharmacology can greatly speed up the process of discovery. Initially, it was science that offered the TV and film world the rudimentary seeds of technology that would fuel creative output. But the TV and film world ran with it — so much so, that the line between science and art is less distinguishable than in any other industry. “We’re getting to a point [on the screen] where we can’t discern anymore what’s real and what’s not,” McCrory notes. “That’s how good Hollywood [technology] is getting. At the same time, you see very little progress being made in scientific visualization.”

What is most perplexing about the stagnant computing power and visualization in science is that modern research across almost all fields is driven by extremely large, high-resolution data sets. A handful of select MRI imaging scanners are now equipped with magnets ranging from 9.4 to 11.75 Teslas, capable of providing cellular resolution on the micron scale (0.1 to 0.2 millimeters versus 1.5 Tesla hospital scanners, at 1 millimeter resolution) and cellular changes on the microsecond scale. The ultra high-resolution imaging provides researchers with insight into everything from cancer to neurodegenerative diseases. While most biomedical drug discovery today is engineered by robotics equipment, which screens enormous libraries of chemical compounds for activity potential around the clock in a “high-throughput” fashion — from thousands to hundreds of thousands of samples — data must still be analyzed, optimized and implemented by researchers. Astronomical observations (from black holes to galaxies colliding to detailed high-power telescope observations in the billions of pixels) produce some of the largest data sets in all of science. Molecular biology and genetics, which in the genomics era has unveiled great potential for DNA-based sub-cellular therapeutics, has also produced petrabytes of data sets that are a quandary for most researchers to store, let alone visualize.

Unfortunately, most scientists can’t allocate dual resources to both advancing their own research and finding the best technology with which to optimize it. As McCrory points out: “In a field like chemistry or biology, you don’t have people who are working day and night with the next greatest way of rendering photo-realistic images. They’re focused on something related to protein structures or whatever their research is.”

The entertainment industry, on the other hand, has a singular focus on developing and continuously perfecting these tools, as necessitated by proliferation of divergent content sources, screen resolution and powerful capture devices. As an industry insider, McCrory appreciates the competitive evolution, driven by an urgency that science doesn’t often have to grapple with. “They’ve had to solve some serious problems out there and they also have to deal with issues involving timelines, since it’s a profit-driven industry,” he notes. “So they have to come up with [computing] solutions that are purely about efficiency.” Disney’s 2014 animated science film Big Hero 6 was rendered with cutting-edge visualization tools, including a 55,000-core computer and custom proprietary lighting software called Hyperion. Indeed, render farms at LucasFilm and Pixar consist of core data centers and state-of-the-art supercomputing resources that could be independent enterprise server banks.

At Northwestern’s CAMI, this aggregate toolkit is leveraged by scientists and visual engineers as an integrated collaborative research asset. In conjunction with a senior animation specialist and long-time video game developer, McCrory helped to construct an interactive 3D visualization wall consisting of 25 high-resolution screens that comprise 52 million total pixels. Compared to a standard computer (at 1-2 million pixels), the wall allows researchers to visualize and manage entire data sets acquired with higher-quality instruments. Researchers can gain different perspectives on their data in native resolution, often standing in front of it in large groups, and analyze complex structures (such as proteins and DNA) in 3D. The interface facilitates real-time innovation and stunning clarity for complex multi-disciplinary experiments. Biochemists, for example, can partner with neuroscientists to visualize brain activity in a mouse as they perfect drug design for an Alzheimer’s enzyme inhibitor. Additionally, 7 thousand in-house high performing core servers (comparable to most studios) provide undisrupted big data acquisition, storage and mining.

Could there be a day where partnerships between science and entertainment are commonplace? Virtual reality studios such as Wevr, producing cutting-edge content and wearable technology, could become a go-to virtual modeling destination for physicists and structural chemists. Programs like RenderMan, a photo-realistic 3D software developed by Pixar Animation Studios for image synthesis, could enable greater clarity on biological processes and therapeutics targets. Leading global animation studios could be a source of both render farm technology and talent for science centers to increase proficiency in data analysis. One day, as its own visualization capacity grows, McCrory, now pushing pixels at animation studio Rainmaker Entertainment, posits that NUViz/CAMI could even be a mini-studio within Chicago for aspiring filmmakers.

The entertainment industry has always been at the forefront of inspiring us to “dream big” about what is scientifically possible. But now, it can play an active role in making these possibilities a reality.

*****************
ScriptPhD.com covers science and technology in entertainment, media and advertising. Hire our consulting company for creative content development. Follow us on Twitter and Facebook. Subscribe to our podcast on SoundCloud or iTunes.