NYP-AILet's Generate
Generative AI - This holiday from the 21st-22nd March 2022, NYP AI hosted its first event solely focused on AI for generation. Using the latest, state-of-the-art deep learning architectures & models, we generated images & music melodies using Artificial Intelligence.
Day 1: Fun with Images
Using AnimeGANv2, we are able to generate an "anime" version from any given image. AnimeGANv2 is a Deep Learning model combining elements of Generative Adversarial Networks & Neutral Style Transfer.
Using pre-trained models from PyTorch, we're able to implement our first GAN under 7 lines of code...
Diving deeper: We explored CLIP, where Natural Language Processing & Computer Vision converges. CLIP is a SOTA deep learning model boasting exceptional capabilities such as zero-shot classification, image similarity search & much more...
Using the CLIP library hosted on Github, we created two models
1. Zero-shot Classifier: Give a list of textual descriptions & have the model decide which description best fits the given image
2. Image similarity search: Finding the most similar image among a bunch of images, given an input image...
Day 2: Music Generation
Ever wondered how to leverage music for machine learning? Here, we created 2 use cases for music in ML...
Using Tensorflow Magenta, a popular Machine Learning library for art, we used...
1. MelodyRNN to continue a given musical sequence by generating subsequent notes
2. Music VAE to interpolate two musical sequences together (Think of it as smoothly transitioning between song A to song B)
Machine Learning is about the data... Hence, we dived into "Note Sequence" at the start. A Note Sequence is a datatype we use for music, which contains a sequence of notes that form a musical melody.
MelodyRNN is an LSTM (Long-Short term memory) model which is used for continuing a given sequence. Apart from generating music, it's also used to generate text, etc...
Music VAE, short for Variational Auto Encoder, allows us to generate a "transition phase" from music A to B.
By simply specifying how long we want our transition to be, we can generate a new piece, combining multiple songs into one...
That's a wrap
Generative AI is one of the coolest, up & coming applications of AI. We hope that you enjoyed learning about it, just as much as we enjoyed creating this content 😉
Until our next event, stay safe!
~ The NYP AI Team
