Yuhao.app Logo

mlx-makemore

Make More ML text-generation with MLX

MLX Make More

WIP. Make More with MLX!

makemore takes one text file as input, and predicts more for you. With a wide range of model selections, such as:

  • Bigram
  • Bigram Neural Network
  • Multi-layer Perceptron (MLP)
  • Convolution Nerual Network (CNN)
  • Recurrent Neural Network (RNN)
  • Long Short-Term Memory (LSTM)
  • Gated Recurrent Unit (GRU)
  • Transform

This project also implemented many functions and modules commonly used in deep learning. Some are out of necessity (e.g. oneHot() and multinomial) as they are missing from MLX. And some are to learn understand how ML frameworks work.

  • oneHot
  • searchSorted
  • multinomial
  • plot
  • CustomLinear
  • BatchNorm1d
  • CustomTanh

Plots

Bigram Frequencies

image

MLP Word Embedddings (C) Principal Components (1&2)

MLP Word Feature Embedding

MLP Learning Rates

MLP Learning Rates

MLP Losses (log10)

MLP Log10 Loss Values

Build Run in CLI

To run in command line, go to the package directory.

Then chmod the helper script:

chmod +x mlx-run.sh

Then:

./mlx-run.sh --package mlx-makemore

Troubleshoot

Try build and run project following CLI instruction.

Alternatively run xcodebuild build -scheme mlx-makemore -destination 'platform=OS X' -derivedDataPath ./.derivedData

Reference

See original python repo by one and only Andrej

Name Commit Messages