Techies on TV
Published in IEEE Spectrum Magazine, Sept 2015
A long, long time ago -- back in the 1970s -- there was a concern within the IEEE about how to raise the public esteem of engineers, who in our opinion didn't seem to be sufficiently appreciated. It was a time when doctor shows were big on television, and an idea was floating around some of our committee meetings: could we get an engineer show on TV?
One of our distinguished engineers suggested, possibly in jest, a show entitled "LA Engineer." What a delightful fantasy that was! But I realized immediately that it would never happen, and even if it did, no one would watch it -- not even us.
But times have changed. I don't think there's much concern now about public esteem for engineers. We're too busy starting companies, changing the world, and stuff like that. And with no impetus from us, there is now a series on HBO entitled "Silicon Valley" all about engineers and computer scientists trying to start a company called "Pied Piper" based on a new data compression algorithm.
OK, so we got a few things wrong with "LA Engineer" -- the venue for one. Silicon Valley and its technology focus and culture didn't exist back then. The other thing: no one ever thought our show would be a comedy. After all, this engineering is serious business. But now we have a hit series on TV that is a comedy about techies. In some ways it's a parody, with caricatures of both the techies and the venture capitalists. Still, the caricatures aren't so broadly overdrawn that we engineers don't recognize familiar personalities and technical and business issues.
Pied Piper engineers grapple with funding, hiring, business strategy, competition, management fads, intellectual property, and legal issues. One thing we don't see, however, is them actually working. That is, after all, boring -- at least probably to most TV viewers.
I got a thrill from hearing mentions of Shannon, Huffman, and Lempel-Ziv compression. Imagine this on a popular TV show! On some occasions we see "LZ" written on their backboard, and another time it's "LZW," giving mention to Welch, who wrote the original compression software and popularized the Lempel-Ziv algorithm.
Of course, the TV techies outdo the known lossless compression algorithms, getting high scores on a fictitious metric called the "Weissman score," which has since become a real measure (see Spectrum, July 2014). This does raise some technical eyebrows, since Lempel-Ziv compression has been shown to be optimum. However, that's asymptotically optimum for a stationary random process, and both "asymptotic" and "stationary" are attributes never actually satisfied in practice.
In compressing written language, for example, LZ algorithms build a dictionary as they go along. However, humans bring a vast amount of understanding and context not present in that dictionary. Moreover, the underlying statistics can change dramatically, say in going from a Shakespeare play to a medical text book. So, yes, it is possible to get better compression than that of known algorithms.
The TV engineers frequently brag that their algorithm is lossless. Other things being equal, lossless is obviously better, but the world today largely lives on lossy algorithms like MP3, JPEG, and MPEG, where small concessions are made in fidelity to enable large gains in compression efficiency. So maybe Piped Piper will crash and burn next season. Stay tuned.