NYP AI: Let's Generate 2022

GitHub - NYP-AI/Learning-Materials: A Public Repository containing all of NYP AI’s past event materials

A Public Repository containing all of NYP AI’s past event materials - GitHub - NYP-AI/Learning-Materials: A Public Repository containing all of NYP AI’s past event materials

GitHubNYP-AI

Let's Generate

Generative AI - This holiday from the 21st-22nd March 2022, NYP AI hosted its first event solely focused on AI for generation. Using the latest, state-of-the-art deep learning architectures & models, we generated images & music melodies using Artificial Intelligence.

Day 1: Fun with Images

Using AnimeGANv2, we are able to generate an "anime" version from any given image. AnimeGANv2 is a Deep Learning model combining elements of Generative Adversarial Networks & Neutral Style Transfer.

Using pre-trained models from PyTorch, we're able to implement our first GAN under 7 lines of code...

Diving deeper: We explored CLIP, where Natural Language Processing & Computer Vision converges. CLIP is a SOTA deep learning model boasting exceptional capabilities such as zero-shot classification, image similarity search & much more...

Using the CLIP library hosted on Github, we created two models
1. Zero-shot Classifier: Give a list of textual descriptions & have the model decide which description best fits the given image
2. Image similarity search: Finding the most similar image among a bunch of images, given an input image...

Zero-shot classification

Day 2: Music Generation

Ever wondered how to leverage music for machine learning? Here, we created 2 use cases for music in ML...

Using Tensorflow Magenta, a popular Machine Learning library for art, we used...
1. MelodyRNN to continue a given musical sequence by generating subsequent notes
2. Music VAE to interpolate two musical sequences together (Think of it as smoothly transitioning between song A to song B)

Machine Learning is about the data... Hence, we dived into "Note Sequence" at the start. A Note Sequence is a datatype we use for music, which contains a sequence of notes that form a musical melody.

Creating notes with code... pretty cool huh

MelodyRNN is an LSTM (Long-Short term memory) model which is used for continuing a given sequence. Apart from generating music, it's also used to generate text, etc...

Music VAE, short for Variational Auto Encoder, allows us to generate a "transition phase" from music A to B.

By simply specifying how long we want our transition to be, we can generate a new piece, combining multiple songs into one...

That's a wrap

Generative AI is one of the coolest, up & coming applications of AI. We hope that you enjoyed learning about it, just as much as we enjoyed creating this content 😉

Until our next event, stay safe!

~ The NYP AI Team