Generating cartoons with LSTM neural networks

The aim of this project is to generate short animation clips with the help of a neural network. I created a small dataset for Tom and jerry using yoloAnnotationTool. I trained Yolov2 classification model on the custom dataset. After the Yolo model was trained I started to process the images. I grayscaled, binarized ,applied a median blur filter then perform an erosion to get the character outline as clear as possible from the image. For unifying the contour used a dilation then created a mask with all the pixels from inside the contour in order to get only the character pixels.

For the neural network I used a LSTM layer combined with a fully connected layer with 6 outputs that represent the x and y coordinates of Tom, along with the png, and the x and y coordinates of Jerry with their corresponding png saved before. I took a random number with the corresponding train data. From that data, the model predicted 20 frames forward.