Port of phi3small (#794)

* start port of phi3small

* fix phi3

* use block sparsity

* compile activation

* nits in readme / mlx lm version
This commit is contained in:
Awni Hannun
2024-05-31 12:54:14 -07:00
committed by GitHub
parent 09aaeac72c
commit 81318ad4a8
5 changed files with 326 additions and 8 deletions

View File

@@ -9,10 +9,10 @@ Some more useful examples are listed below.
### Text Models
- [MLX LM](llms/README.md) a package for LLM text generation, fine-tuning, and more.
- [Transformer language model](transformer_lm) training.
- Large scale text generation with [LLaMA](llms/llama),
[Mistral](llms/mistral), [Phi-2](llms/phi2), and more in the [LLMs](llms)
directory.
- Minimal examples of large scale text generation with [LLaMA](llms/llama),
[Mistral](llms/mistral), and more in the [LLMs](llms) directory.
- A mixture-of-experts (MoE) language model with [Mixtral 8x7B](llms/mixtral).
- Parameter efficient fine-tuning with [LoRA or QLoRA](lora).
- Text-to-text multi-task Transformers with [T5](t5).
@@ -21,7 +21,7 @@ Some more useful examples are listed below.
### Image Models
- Image classification using [ResNets on CIFAR-10](cifar).
- Generating images with [Stable Diffusion](stable_diffusion).
- Generating images with [Stable Diffusion or SDXL](stable_diffusion).
- Convolutional variational autoencoder [(CVAE) on MNIST](cvae).
### Audio Models