The Growing Matrix
a personal computing device that grows its size to activate bigger models
navigating public transit
context window: 1024
embedding: 8192
attention key-query: 256
attention value: 48
feed-forward layer: 96
17B parameters
helping with physics homework
context window: 768
embedding: 32768
attention key-query: 128
attention value: 48
feed-forward layer: 128
67B parameters
debugging software project
192B parameters
context window: 8192
embedding: 32768
attention key-query: 2048
attention value: 768
feed-forward layer: 512
