John R. Pierce, 1980 [1961]
Claude E. Shannon and Warren Weaver, 1949
[1998]
(June 2013)
 |  |
One of these books is a very dense, technical
work, heavy on the jargon and academic prose and littered with equations and
diagrams that are very difficult for the layperson to get their head around.
And the other is worse.
So I finally committed to reading the damn
things. The Pierce book seems to be widely regarded as the standard overview
and obviously the Weaver/Shannon volume is the Patient Zero of the whole sorry
mess.
The maths started off as just about barely
comprehensible and then very quickly accelerated away from my grasp. Conceptually
the learning curve was initially less steep. The first few chapters of Pierce
were testing but interesting and manageable; then he starts talking about multi-dimensional
hyperspheres and writing perfectly cromulent sentences like the one below. Suddenly
that learning curve turns into a cliff; the ground disappears and you’re left
momentarily hovering in mid-air with your conceptual legs whirring away before realisation
hits, leaving your intellect with barely enough time to look to camera and hold
up a handpainted ‘Help!’ sign before the plummet commences.
“The noise figure is the ratio of the total output noise, including
noise due to Johnson noise for a temperature of 293° Kelvin at the input and noise produced in
the receiver, to the amplified Johnson noise alone.”
This is a quantum sentence. Dealing as
Information Theory does with ideas of binary encoding and the extraction of
meaning and order from chaos and entropy, it’s entirely appropriate that we
should end up with sentences like this; a sentence that flips wholly between
complete lucidity and utter incomprehensibility with each successive rereading
without passing through any intermediate states in between. Clearly the only
sensible thing to do is to shut it in a box with a radioactive trigger and some
poison and never open the lid again. Or at least until you’re sure it’s dead…
I shall, in due course, be putting up some
more constructive thoughts over at the other place, but it’ll take a while to
boil them all down into anything even approaching coherence or applicability. Like
the cat, I wouldn’t hold your breath.
I once had a seminar course in Hegel's "Phenomenology of Mind"... As an undergrad you just figure you are stupid when you read that; however, once I read secondary works on it, I realized he was a spectacularly bad writer, with some interesting but wrong-headed notions. Lesson? Read a precis when the going gets tough. Life's too short.
ReplyDeleteInterestingly, I've been reading almost the exact opposites of precis(es?) here, in that as each volume (read a couple of other books on this a while back) has got less and less pop-science and more technical, they've been getting shorter and shorter. Makes you realise just how dense the original was. In case that wasn't immediately obvious just from looking at it. Although otherwise you make a very valid point.
DeleteHey, nice!... Signals and noise sounds like you'll be dealing with Fourier transforms and such. Good stuff!
ReplyDeleteThought this might be up your street. Yep, they feature, but I'm more taken with the possibilities of Zipf's Law and Markov chains. I instinctively feel they could be quite powerful ways of explaining important things about language, but can't quite nail it down to specifics, or at least ones I can actually use.
DeleteOf course it's just as likely I'm being seduced simply because they have funny foreign names.
I know how you feel as I was in the same position a few years back. Certainly Pierce and Warren Weaver's intro to Shannon's original work are much more accessible than Shannon's work by itself depending on your level of mathematical sophistication. Read them a couple of times while skimming through the equations. Then take out a pencil and paper and work your way through some of the equations. This will make things infinitely easier going as most (if not all) mathematics isn't meant to be read quickly and easily like a novel.
ReplyDeleteIf it helps, Khan Academy has some intro videos on information theory: https://www.khanacademy.org/math/applied-math/informationtheory/info-theory/v/intro-information-theory
Thanks for the comment and the pep-talk.
DeleteI'll be honest, the maths is way down the road at the moment. At the moment It's all about how I can make the ideas work for me. There may come a point where I need to plug in some numbers but that's probably a long, long way off.