When sound hits an object, it causes small vibrations of the object’s surface.
We show how, using only high-speed video of the object, we can extract those
minute vibrations and partially recover the sound that produced them, allowing
us to turn everyday objects—a glass of water, a potted plant, a box of tissues,
or a bag of chips—into visual microphones. We recover sounds from highspeed
footage of a variety of objects with different properties, and use both real
and simulated data to examine some of the factors that affect our ability to
visually recover sound. We evaluate the quality of recovered sounds using intelligibility
and SNR metrics and provide input and recovered audio samples for direct comparison.
We also explore how to leverage the rolling shutter in regular consumer cameras
to recover audio from standard frame-rate videos, and use the spatial resolution
of our method to visualize how sound-related vibrations vary over an object’s
surface, which we can use to recover the vibration modes of an object.
Before you develop your app, it’s important to become familiar with the technical, content, and design criteria that we use to review all apps. We’ve highlighted some of the most common issues that cause apps to get rejected to help you better prepare your apps before submitting them for review.