The goal of this cluster is to introduce students to computer science principles and applied psychoacoustics with an introduction to sound design for mobile devices and an emphasis on creating 3D sound environments for video games and other applications. Students will study the physics of sound, field recording techniques, analysis and editing. Students will learn Apple’s new computer language Swift, Xcode and the AVFoundation framework.
The course introduces students to sound programming and the latest iOS technology. Working in groups within a unique lab setting, students will collaborate to design, build and complete a sound oriented iOS app while learning to create environmental field recordings of various sound sources. Using open source software tools to perform waveform and spectral analysis to “see” the components of sound, students will learn the basics of digital audio manipulation and sound processing using Apple’s (Audio Units) AU and the AVFoundation.
“Zoom is an iOS app that uses gyroscope data and touch gestures to change the parameters of a physical model of the sound of a ping pong ball. The project was created using the Swift programming language in Xcode. Physical models were realized using the Chuck programming language and the 3D elements were coded using iOS’s SceneKit”.
“Using the Fast Fourier Transform to analyze audio from the device’s microphone, Pitch Perfect maps frequency and volume to parameters that change the behavior of a sprite created with iOS’ SpriteKit. Physically modeled sounds are triggered according to the octave of the pitch. Volume is mapped to the size of the sprite and pitch is mapped to color hue. Sounds on this app are meant to help people with stress dissorders. Physical models were created using the Chuck programming language”.
“Bubble Beats is an iOS app that combines the physical models of the different sounds produced when popping bubble wrap. In addition, sounds can be tranfosrmed by using different filters. The physical model and filters were created using the Audikit synthesis library for iOS”.
“The purpose of iBounce is to give people who are not musically trained and opportunity to explore and created music within a fun and interactive app. Sounds were synthesiszed using sinusoidal oscillators, filters and envelopes using the Chuck programming langugage. In iOS, a physics world was created to control a ball using the accelerometer of the device. The goal of the app is to hit different parts of a bounding rectangle to “play” different melodies”.
“Shake it Off incorporates the gyroscope sensor to detect motion along the X, Y and Z axes to play the sounds of real objects that were recorded interacting with different materials. To understand the acoustic response of the different materials, spectrograms were used for analysis to create physical models in Chuck as part of the research portion of the project”.