Five Forms Of Filtering

We create economic value out of information when we figure out an effective strategy that includes aggregating, filtering and connecting. The three steps interact and reinforce each other – and successful information-based business models have all three. We can undertake business model innovation by changing our methods in these three areas, or by changing where in the value network the processes take place. I've run across a few things recently that have gotten me thinking about filtering – and it made me realise that we have another classification issue here. Here is how Clay Shirky frames it:

So, the real question is, how do we design filters that let us find our way through this particular abundance of information? And, you know, my answer to that question has been: the only group that can catalog everything is everybody. One of the reasons you see this enormous move towards social filters, as with Digg, as with del.icio.us, as with Google Reader, in a way, is simply that the scale of the problem has exceeded what professional catalogers can do. But, you know, you never hear twenty-year-olds talking about information overload because they understand the filters they're given. You only hear, you know, forty- and fifty-year-olds taking about it, sixty-year-olds talking about because we grew up in the world of card catalogs and TV Guide. And now, all the filters we're used to are broken and we'd like to blame it on the environment instead of admitting that we're just, you know, we just don't understand what's going on.

Filtering is what helps us deal with the vast amount of information available to us. We try to filter information so that we end up with something that is relevant to us – it helps us learn something, it helps us solve a problem, it helps us develop a new hypothesis about the world around us. These are all connections – and this is what really drives value creation. However, we can't connect without some filtering going on. So filtering is important, and it's a term that includes several different sub-types. I can think of at least five forms of filtering.

The five forms of filtering break into two categories: judgement-based, or mechanical.

Judgement-based filtering is what people do. At its most basic level, we have naive filtering. This is what we do when we don't know anything about the information that we are trying to filter. This is a fairly complex internal process, and there are plenty of models available for what is happening in this step. It's basically everything going on in the 'Sense' step in this diagram by Harold Jarche:

As we gain skills and knowledge, the amount of information we can process increases. If we invest enough time in learning something, we can reach filter like an expert. I previously explained how this process can work in bird-watching.

However, even experts can't deal with all of the information available on the subjects that interest them – that's why they end up specialising. One way to increase the amount of information that gets filtered is by relying on a network. This can be a network of learners, as in the Connectivism course run by George Siemens and Stephen Downes, it can be a group of people with a similar interest, like all of us talking about #innovation on twitter, it can be large groups of otherwise unconnected people as in Shirky's examples on places like Digg and Delicious. Networks expand our reach enormously.

There can also be expert networks – in some sense that is what the original search engines were, and what mahalo.com is trying now. The problem that the original search engines encountered is that the amount of information available on the web expanded so quickly that it outstripped the ability of the network to keep up with it. This led to the development of google's search algorithm – an example of one of the versions of mechanical filtering: algorithmic.

Algorithmic filtering is used by most of the filtering tools available on the web. They can cover the entire web, like google does, or sub-sections of it, like an RSS feed does. Howard Rheingold describes many of these sorts of tools in his post and video on Mindful Infotention.

Rheingold also provides a pretty good description of the other form of mechanical filtering, heuristic, in his piece on crap detection. Heuristic filtering is based on a set of rules or routines that people can follow to help them sort through the information available to them.

Why is filtering important? Understanding the variety of filters available explains why there are often arguments about the discrimination process, and over what role a particular filter plays. If someone writes about filtering, and they mean 'algorithmic filtering', a reader thinking of filtering in terms of 'network filtering' is likely to misunderstand the discussion. Another source of confusion is that some people talk about filtering not as a search for useful information, but as a way to block information that annoys them.

Filtering by itself is important, but it only creates value when you combine it with aggregating and connecting. As Rheingold puts it:

The important part, as I stressed at the beginning, is in your head. It really doesn't do any good to multiply the amount of information flowing in, and even filtering that information so that only the best gets to you, if you don't have a mental cognitive and social strategy for how you're going to deploy your attention. (emphasis added)

Filtering and connecting is what leads to important skills, like pattern recognition (described well by Venessa Miemis).

Finally, we can use these ideas about filtering to help with business model innovation by changing where it takes place in the value network. One of Shirky's points is that since Gutenberg, the economic logic of publishing required publishers (of books, music, movies) to act as filters in order to maximise their investment. As publishing and filtering has shifted out to human networks, publishers no longer need to fill this role. Someone (or some network) needs to, and since that creates value, it's something that can perhaps be monetised.

You can see this in investing. You can put money in Berkshire-Hathaway, where investment choices are run through the personal expert filter of Warren Buffett. Or you can invest in individual stocks recommended by a broker- which is filtering through an expert network. Or you can take advantage of DIY investing, where you do your own filtering, probably aided by some heuristic filters as well. Three different investing business models based on three different filtering methods.

People or networks filling the filtering role now are creating significant value – and people trying to come up with innovative business models in these fields should be thinking about how they can create value through filtering. Of course, the filtering needs to be part of an overall aggregate, filter and connect strategy – which is at the core of successful information-based digital business models.

(Thanks to John, Phil Long & Nancy Pachana for talking to me about these ideas- of course, none of this is their fault. Special thanks to Phil for editorial suggestions.)

Thanks to Tim Kastelle / Tim Kastelle Blog / Innovation Leadership Network Blog
http://timkastelle.org/blog/2010/04/five-forms-of-filtering/