Cogs Bad
There is something really wrong with modern programmers. Very wrong indeed.
Mailinator creator Paul Tyma has a great blog post on how he compresses our email by 90%. He has a simple LRU cache of lines and consecutive lines from emails, so emails become a list of line string pointers where most are shared. He also has a background thread doing rather stronger LZMA compression on large emails. He’s winning.
(Its well worth reading the mailinator blog post again and spend a minute thinking about how to store the individual lines from 3500 emails per second; how you’d need to fetch them to recover an email if someone wanted to read it and so on. It turns out to be quite obvious that a normal Java hash-table and linked list is a just fine way of doing it with excellent performance characteristics ;)Thing is, in one of the comments on the Hacker News version of the article, someone wonders why he didn’t use Redis for the LRU.
I think of the kindest, least shouting way to hint that that’s not such a good idea; I say it might be higher latency to block on a Redis LRU than using a in-process data-structure. Other commenters responds by saying that Redis is known to be fast; that they have just chosen node.js and Redis for their startup because of its performance.This is wrong on so many levels. So many levels. I don’t single these people out - they are just a useful illustration. Their mindset is endemic in this industry. All around you, the new generation of programmers are making the same assumptions.
