synopsisinternal representationcompartmentalizationmatrixconcept product

The Growing Matrix

a personal computing device that grows its size to activate bigger models

navigating public transit

context window: 1024

embedding: 8192

attention key-query: 256

attention value: 48

feed-forward layer: 96

17B parameters

Navigating public transit

helping with physics homework

context window: 768

embedding: 32768

attention key-query: 128

attention value: 48

feed-forward layer: 128

67B parameters

Physics homework

debugging software project

192B parameters

context window: 8192

embedding: 32768

attention key-query: 2048

attention value: 768

feed-forward layer: 512

Debugging software

see how they come together as a concept product →