Machine Learning for Artists
Course Number: IM-2119
Prof. Aaron Sherwood
NYUAD Spring 2018
Location: Abu Dhabi
Classroom: C3 Rm. 153
Meeting Time: Monday 10:25am - 1:05pm Wednesday 11:50am - 1:05pm
Email Address: [email protected]
Office: C3 Rm. 163
Office Hours: By Appointment
Credits: 4
Prerequisites: Students should have taken at least one of the following courses - Intro to IM (IM-UH 1010), Mashups (IM-UH 2110), Politics of Code (IM-UH 3110), Intro to Computer Science (CS-UH 1001), Decoding Nature (IM-UH 2118) or request specific instructor approval.
This course counts toward the following NYUAD degree requirements:
- Concentrations > Multidisciplinary Concentrations > Interactive Media and Technology
Course Description:
Artificial intelligence and machine learning algorithms affect many aspects of our lives whether we realize it or not: banking transactions, healthcare treatments and diagnoses, entertainment recommendations, smart car functionality, customer service agents, financial trading… the list goes on and on. The power of these algorithms lies in their ability to leverage computers to "study" and "learn". Instead of programming a computer to do a specific task, we program the computer to train itself how to do any number of tasks. As artists, how can we harness the power of these algorithms and apply them towards creative endeavors? This class will explore that basic question. Through a combination of high level applied machine learning techniques, speculative design of artificial intelligence, and some basic understanding of how these algorithms work at a low level, students will explore this rich new field. With their machine counterparts, they will create images, sounds, text, intuitive interactions, chatbots, and more.
Learning Outcomes:
- Foster an ability to think critically about AI in terms of design, ethics, and global impact
- Develop a speculative artistic vision of the future that leverages both critical perspectives and creative applications of emerging AI technologies
- Create intuitive interactions using machine learning
- Understand some of the low-level math and code used in neural networks and deep learning
- Use Python, Tensorflow, & Keras to implement machine learning algorithms
- Use various types of neural networks to generate sound, visuals, and text
Teaching Methodologies:
This is a production based course. Students will be learning primarily through hands-on experience and iterative experimentation. Classes will move quickly through each topic, first providing a theoretical foundation along with technical instruction, and then encouraging students to learn through play, experimentation, collaboration, and exploration. Individual classes will include lectures, discussions, examples, workshops, group work, and presentations. Outside of the class, students will engage in both individual and group work.
Course Expectations:
Students can expect to apply machine learning algorithms to the creative process. There will be four major production assignments, each focusing on different areas of creation. There will also be regular exercises throughout the semester in order to develop the necessary skills for the course.
Class Participation:
Class participation will be expected during discussions, presentations and project critiques. Showing up on time, engaging in the class discussion, and offering advice and input in the class is an important part of the class. Participating in class discussions is helpful for me to get to know you as an individual and keep track of your progress, but most importantly, it provides you and your classmates with the opportunity to share failures, successes, and insights on the work you are doing.
You are expected to show work in class. This includes working prototypes, failed assignments, things that don't work the way you expect, and so forth. The shorter of the two classes in a week is given over to your work, expect to be called on and show something. Don't be afraid to volunteer to show what you did, or failed to do.
If you do not ask questions, I can only assume you understand the material completely. Asking questions about concepts you do not understand and showing work that did not function as expected is not a sign of failure, it is an opportunity to learn.
Assignments:
Each student will thoroughly document their work online. All assignments are due the following class. Usually, production assignments will be assigned on Wednesdays and will be due the following Monday. Every production assignment should be documented with photos/videos/sketches/audio/etc (see get_started.md for more info). For all reading assignments, students are expected to write a short blog post (approx. 100-200 words) with their reactions, prior to the class discussion on the material.
In terms of nomenclature for the production assignments, there will be smaller weekly assignments called Exercises. There will also be four larger assignments, which will be called Projects. These larger Projects are worth a considerable portion of the overall grade, and subsequently, more effort will be expected for these assignments.
Graded Activities:
- Attendance: 10%
- Participation: 10%
- Documentation and Reading Responses: 20%
- Exercises: 20%
- Projects: 40%
Grading Rubrics
Production assignments will be graded on the overall quality of work, comprehension of topic, how well project is functioning, and individual capacity/ability improvement (are you getting better compared to yourself). Documentation/Reading will be graded on critical reasoning and erudition of assigned topic if reading assignment, and quality & extent of documentation if production assignment. Participation will be graded on a combination of class discussion, group project work, office hours meetings, and possible extracurricular class trips/workshops/talks/movie screenings/etc.
Attendance:
Attendance is mandatory. Unexcused absences and lateness will negatively impact your final grade for the class. Two late arrivals will equate to one unexcused absence. If you're going to be late or absent, please email me in advance. If you have an emergency, please let me know as soon as you can.
Laptops:
Laptop use is fine if you are using your laptop to present in class, or if we're in the middle of an exercise that makes use of it. Whenever classmates are presenting or we're in the midst of a class discussion, please keep your laptop closed. The quality of the class depends in large part on your attention and active participation, so please respect that and close your lid.
Mobile Phones:
Please put them on vibrate or turn them off before you come to class. If you have an emergency that requires you to answer your phone during class, please tell me ahead of time.
Academic Integrity:
As set forth in NYU Abu Dhabi's Academic Integrity Policy, the relationship between students and faculty at NYU Abu Dhabi is defined by a shared commitment to academic excellence and is grounded in an expectation of fairness, honesty, and respect, which are essential to maintaining the integrity of the community. Every student who enrolls and everyone who accepts an appointment as a member of the faculty or staff at NYU Abu Dhabi agrees to abide by the expectation of academic honesty. The full policies and procedures relating to Academic Integrity may be found on the NYUAD Student Portal: https://students.nyuad.nyu.edu/campus-life/student-policies/community-standards-policies/academic-integrity/
Major Projects:
- Interactive Project
- A. I. Speculative Design Project
- Visual Project
- Sound Project
Bird's Eye View:
The semester is organized roughly as follows:
- Weeks 1-3
- Focus on using high level ML tools to understand deeper ML principles
- Create intuitive interactions with ML
- Weeks 4-8
- Look at some of the math and code behind ML algorithms
- Speculative Design Project
- Weeks 9-14
- Applications of various types of neural nets & deep learning for creative purposes
- Visual Project & Sound Project
Required Text:
- Rashid, Tariq (2016). Make Your Own Neural Network. CreateSpace Independent Publishing.
Weekly Schedule:
- What is Artificial Intelligence - Machine Learning?
- Overview
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
- Classification 1
- Decision Tree
- K-Nearest Neighbor
- Intro to Wekinator
- Readings
- Creative AI: https://medium.com/@creativeai/creativeai-9d4b2346faf3
- What is AI & History: Chapter 1 Artificial Intelligence a Modern Approach
- Regression
- Linear/Plynomial
- Neural Net (soft intro)
- Classification Pt. 2
- Naive Bayes
- Ada Boost
- Support Vector Machine
- Probability
- Multiple Classifiers
- Readings
- AI Revolution Parts 1 & 2
- Play Paperclips
- Dynamic Time Warping
- Various sensing methods
- Vision - OpenCV
- Fiducials
- Frame Differencing
- Optical Flow
- Audio
- RMS
- FFT
- Spectral Centroid
- Mel-Frequency Cepstral Coefficients
- Maximilian
- http://www.sonicvisualiser.org/
- Vision - OpenCV
- Readings
- Ethics of Machine Intelligence
- Filtering Dissent
- Breaking The Black Box - Parts 3 & 4
- Facial Recognition
- Intro to Neural Networks
- The Perceptron
- Readings
- Linear Algebra Primer
- Vectors
- Matrices
- Activation Function
- Sigmoid
- Calculus Primer
- Derivatives
- Power Rule
- Chain Rule
- Gradient Descent
- Reading
- Make Your Own Neural Network, Part 1 - How They Work, Tariq Rashid
- MNIST
- Python: Jupyter, IPython, Numpy, SciPy, etc.
- Reading
- Deep Learning History: http://www.andreykurenkov.com/writing/a-brief-history-of-neural-nets-and-deep-learning/
- Intro to Tensorflow
- Intro to Keras
- Clustering
- Autoencoders
- Generative Adversarial Networks
- Convolutional Neural Nets
- Image Classification - CIFAR-10 ( online demo)
- deep dream
- style transfer
- photo realistic style transfer
- pix2pix
- Compositional Pattern Producing Network
- GAN - VAE
- t-sne
- Recurrent Neural Nets
- LSTM
- GRU
- Applications of RNN
- Generating Text
- Readings
- Magenta (RNN)
- wavenet (CNN)
- nsynth ( https://experiments.withgoogle.com/ai/sound-maker)
- Organizing sound
- Commercial
- Music Information Retrieval
- Voice
- Reading