It would probably have, were this blog post not almost a direct response to Three Blue, One Brown's latest video series on exactly this topic. What's strange is that the content and structure of...
In this post, I’m going to walk us through the basics of compression to understand its deep relationship with language modeling. It’s probably going to blow your mind.
What's strange is that the content and structure of the blog post is very similar to that of the video, which was released roughly two months ago, and yet no explicit mention is made of it?
This, coming from an organisation that seems to be all-in on AI, is putting a bad taste in my mouth..
Oooh, I like these visualizations a lot, thanks! The titular fact is why there's still an open prize to compress the contents of Wikipedia. Also, another way to phrase "how small we can make our...
I was reading about compression recently when I stumbled upon something crazy: that compressors and LLMs are, at their core, trying to solve the exact same problem.
In this post, I’m going to walk us through the basics of compression to understand its deep relationship with language modeling. It’s probably going to blow your mind.
It would probably have, were this blog post not almost a direct response to Three Blue, One Brown's latest video series on exactly this topic.
What's strange is that the content and structure of the blog post is very similar to that of the video, which was released roughly two months ago, and yet no explicit mention is made of it?
This, coming from an organisation that seems to be all-in on AI, is putting a bad taste in my mouth..
Oooh, I like these visualizations a lot, thanks!
The titular fact is why there's still an open prize to compress the contents of Wikipedia. Also, another way to phrase "how small we can make our entropy?" is kinda like asking "what is the Kolmogorov complexity of our data?".
I wrote something much-more-stupid and much-less-pretty about this topic a while back; algorithmic information theory is really awesome.
From the article: