Time: Mon and Thurs, 11:45am-1:25pm Location: Ryder 233 Instructor: Victor Zappi -- [email protected] Office: Ryder 357 Office Hours: Tue and Fri, 3-4pm (or any other available time/day by appointment)
This course proposes to explore embedded computers and their employment for the design of interactive audio applications. The main objective of the course is to teach how to program audio applications that takes advantage from the unique features of these systems; this will be accomplished by transferring the coding skills acquired so far on general purpose computers to the languages and the programming rationales of embedded hardware. The classes will primarily cover low-latency real-time audio synthesis/processing, yet students will go through an in-depth introduction to physical computing, sensors and simple electronics. The final project will provide students with a concrete opportunity to gain experience with the full design development-test cycle of software and hardware interactive audio technologies.
By the end of the course, students should be able to: • discuss the features and the limits of both embedded systems and general purpose computers; • analyze the requirements of a target audio application, relate them to human physiology and choose the
most appropriate technology for its design;
- combine creativity and computational thinking in the design of responsive audio applications;
- design simple custom electronics and interface it with embedded audio software;
- transfer high-level programming skills to the specific context of low-latency embedded systems (using Pure Data);
- develop high-performance digital signal processing and interactive applications in C++;
- leverage optimization techniques and lateral thinking to deal with limited computational resources (using Assembly) and physical constraints;
- prototype self-contained, highly responsive musical instruments and interactive audio installations/devices.
There is no required text book for the course. Yet, the topics presented throughout the classes are covered in the following books:
Kuo, Sen M., Bob H. Lee, and Wenshun Tian (2013). Real-time digital signal processing: fundamentals, implementations and applications. John Wiley & Sons, 2013. ISBN: 978-1-118 41432-3
Molloy, Derek (2019). Exploring BeagleBone: Tools and Techniques for Building with Embedded Linux, 2nd Edition. Wiley. ISBN: 978-1-119-53316-0
You must have your own laptop(+headphones), with installed Pure Data, which is a free, cross-platform, open-source software. You will borrow a kit from the Department, composed of a Beaglebone Black embedded computer, a Bela cape and a set of electronic components and sensors. The complete kit shall be returned after the final project presentation (Proto Fest).
Some of the assignments as well as the final project may require you to purchase some affordable equipment/ material, including additional electronic components, sensors, actuators.
Date M 1/6
Unit Unit 1: Introduction to Embedded Computers
Topic Embedded Linux, Scheduling, Audio and Control Latency
Due
T 1/9
Unit 2: High-level Embedded Audio Programming (Pure Data)
Bela Setup and Introduction to Embedded Pure Data
M 1/13
Midi and OSC Control
M 1/20 T 1/23
Martin Luther King Unit 2 (continued)
King Jr.'s Birthday (NO CLASS) Computational Power Analysis, Full Duplex and Audio Latency Analysis
Assignment 1
M 1/27 T 1/30
Unit 3: Electronics and Physical Computing (Pure Data)
Circuits, Sensors and Analog Inputs Digital Inputs and Control Latency Analysis
M 2/3
Actuators, Analog and Digital Outputs
M 2/10
Processing on Embedded Computers (C++)
Full Duplex, Filtering and Processing
M 2/17
Presidents'
Presidents' Day (NO CLASS)
M 2/24
Applications (C++)
Outputs and Actuation
M 3/2
Spring Break (NO CLASSES)
M 3/9 T 3/12
Project Kick-off Unit 6: Performance and Optimization (C++)
Kick-off Mentoring Computational Power and Latency Analysis
Project proposal
M 3/16
Vectorized Operations (NEON Library)
M 3/23
IN-CLASS WORKSHOP
WORKSHOP (project development mentoring)
M 3/30
Audio Programming (Assembly)
Vectorized Operations (Custom Routines)
M 4/6
NO CLASS (project development)
M 4/13
NO CLASS (project development)
Your final grade will be based on the following:
- Attendance and participation (including TRACE Course Evaluation)
- Assignment 1 on high-level audio programming (fixed)
- Assignment 2 on high-level audio programming (open-ended)
- Assignment 3 on embedded digital signal processing (fixed)
- Final project proposal
- Final project demo at Proto Fest
Assignment 1 will consist of developing a Pure Data patch that runs on Bela and fulfills a set of simple structural, synthesis and control constraints as imposed by the instructor.
Assignment 2 will be an open-ended small project; you will be asked to design and develop a more advanced synthesis/processing Pure Data patch and interface it with simple custom circuitry (sensors and/or actuators). Assignment 3 will consist of recreating a real-time digital signal processing application on Bela using C++; the application will be chosen by the instructor.
For the final project, you will have to design and develop a real-time audio application (e.g., musical instrument, effect processor, audio installation) that capitalizes on the specific features of embedded audio technologies. You will submit your code and present the project as part of a small Interactive Media exhibition, called Proto Fest, open to all CAMD students, faculty and staff. During the event, the instructor will request a formally evaluated demo of the project, in the form of:
- an interactive showcase of the device's functionalities;
- an analysis of its inner workings;
- a discussion of the extent to which it leverages the embedded audio architecture.
Slides and other additional material are not requested, but feel free to organize the demo as you prefer.
A 2-3 page project proposal will have to be submitted around a month before Proto Fest. It will have to include the details of the device design (what it does and how), a list of required materials and a development plan (how hardware and software development is scheduled). You are encouraged to include pictures, sketches, code snippets and any other material that may help describe the device and the underlying technologies.
The day of the proposal submission, I will meet each of you in class for a short private meeting (Project Kick-off Mentoring), to help you set off the development of the project (e.g., material gathering and scheduling advice). Two of the following classes will be dedicated to the development of your projects, in the fashion of workshops during which you will get additional advice from me and from your peers.
Plagiarism will result in a 0 grade. The code you submit for your assignments and final project will be scanned by an automatic plagiarism checker, and I will personally check the authenticity of the design of your final project. There will be no make-ups in case of plagiarism.
The College of Arts, Media and Design considers student feedback essential and requires all students to complete TRACE evaluations at the end of the semester. You will be asked to provide a screen shot to your instructor that reflects your participation. Note that you can, anonymously, opt out of completing the survey and still obtain the screen shot that satisfies the TRACE requirement. The TRACE evaluations will be included in the Attendance and Participation portion of the final grade for the course.
To enroll, you must have taken MUST 2341--Computer Music Fundamentals. Other courses on general purpose programming and Physical Computing (e.g., ARTG 2260, ARTG 3250, EECE 2150, EECE 2160, EECE 2300, EECE 2560) may be accepted as prerequisites, but this is at the discretion of the instructor.
MUST 3540 follows a similar policy as the music theory courses at Northeastern. Two absences (for 100 minute classes) are allowed. On the third absence and each thereafter, you'll be assessed a 5% deduction for each absence.
The instructor encourages you all to join Northeastern's commitment to create a friendly, respectful and inclusive environment, inside and outside the class. Critical thinking and constructive discussion will be fostered and positively evaluated, while any kind of discrimination and harassment will not be permitted. In case you feel the need for special accommodations, because of disabilities, emergencies or any other situation that may affect your learning experience, please contact the instructor as soon as possible. We will discuss appropriate solutions that might be helpful to you. Timeliness is extremely important though! Just contact me, even if you are unsure about the impact of the problem. The more you wait the less I can/will do.