Network Compression

Human beings are (among other things) compression machines. Every day we are confronted by massive amounts of information, for which we only can deal with a small fraction. Humans take those fractions, snippets of data over time, and construct a seamless mental narrative. We decompress these snippets so effortlessly that the gaps in our information stream never seem to appear.

Take the simple act of blinking. The average person blinks 15-20 times per minute. Many of us were taught that this autonomic act is there to help us spread lubricants across our eyes to keep them from drying out. However, the frequency of blinking is far more than is required for moist eyes. The math shows our eyes are closed due to blinking for about 10% of our entire day. Think about that: in the last hour you did not see anything for about 6 minutes, and I’ll bet you never noticed.

Science is starting to show that we use the blink as a mental “rest,” turning off sight so that we can process the information mentally. Visually, our brains are constantly discarding data not deemed necessary for survival. Something moving across the visual field immediately gets our attention, but we gloss over the detail in a static image. A repeated touch, smell or sound can happen enough for it to no longer be novel, and we quickly learn to ignore it. You may have noticed how overstimulation of any sense can be overwhelming to the point where we shut down it down—an effort to reduce its cognitive load. In each of these instances and hundreds more, we are absorbing high bandwidth data and filtering it down to something that can be used to reconstruct a mental model of our surroundings. Show this human algorithm to a computer scientist, and they will tell you that that is data compression (actually they’d say it is “lossy compression,” but that’s another topic altogether). The thing about humanity is we are compressing all the time; it is a central fact of our survival. We even do it with friends.

There is a somewhat famous number in social networking theory called “Dunbar’s Number.” Dunbar’s Number was hypothesized in the 1990s by anthropologist Robin Dunbar who spent a lot of time watching monkeys team up in groups. As a result of his studies, Dunbar speculated that the practical limit on the number of stable relationships any one of us can have is about 150 people. Put another way, our brains can only maintain relationships with 150 friends. Now go back and look at how many “friends” you have on FaceBook! Note that Dunbar is not talking about mere acquaintances, but actual friends—a cohesive group.

Based only on my own experience, the effort required to keep up with 150 close friends seems exhausting. Do I know 150 people who might be friends? Sure. Can I construct, only in my mind, a network of 150 people I care about enough to be engaged with over time? Yes. Yet, how do I do this without it becoming a constant (and tiring!) daily activity? The key is network compression. I do not need to know the details every day of every one of those 150 friends, I only need to engage with a fraction of that number. From that fraction, I can easily reconstruct whole portions of my more substantial network.

Think about it. Almost every one of us has a couple of critical friends who are super “tapped in”—they’re the first person we reach out to when we want to figure out what’s happening. We contact that key friend and that “node” in our network helps us reconstruct that portion the group of friends they, in turn, are connected to (a term used by network theorists is that our key friend has “high betweenness centrality,” but that’s another topic too). The fact is, (and having looked at the literature I can’t find anyone else saying this or I’d cite it) our compressed network topology is not 150, it’s probably more like 20 or 30 or even less. That group is a “compression” of a much larger group—I’d guess we all have an uncompressed number far more extensive than 150. In this way we’re always 2-3 degrees of separation from our whole network — this is much less of a cognitive load. The problem is that over time that network becomes a highly filtered input into our worldview. It becomes possible over time for us to have lots of friends and actually know less about what is really happening around us.

Which brings us back to why you should care to spend time with those you might not otherwise want to. You see, over time your network has evolved to compress even the data it shares amongst itself. Do you have a friend who can finish your sentences before you do? How about get-togethers where you are sharing old war stories? Can you accurately guess where one of your close friends will come in on a debate? Do your friends know how to push your buttons, or know when to call just when you need it? Are there some things “you just don’t talk about?” These are all the behavioral outputs of compression. Over time, we get less and less uncompressed data the more time we spend exclusively in our network. In effect, our worldview becomes increasingly curated (compressed) by a social network purpose-built to reduce bandwidth while reinforcing pleasure and survival. Over time, we learn less and less about the “truth”—only learning the truth we have built a network to see. This is self-reinforcing because our network feeds back upon itself. Wonder why internet memes work so well? They circulate and amplify in social networks designed to “see” them.

Friends are great, I am not advocating getting rid of them. I am, however, a big advocate of going to the edge of my network and learning something different. It is worth the cognitive load that comes with getting uncompressed data. In this context, “cognitive load” is the uncomfortable feeling we all get when introducing ourselves, or making a new friend. Try it. Be random. As your Lyft driver how business is going and then listen. Buy someone you know “only a little bit” lunch and get to know them a bit better. Invite a new face into a regular weekly meeting. Volunteer somewhere. Join a book club. You don’t have to do this all the time, but make network discomfort a part of your routine by going to the edge. You might be surprised at what you’ve been missing.