Information Overload and Managing Content Filters

Since Clay Shirky’s famous remark about filter failures, we know that filters have a direct effect on the quality and quantity of information we end-up consuming. Personally, I’ve never wholeheartedly agreed with the belief that there is a systematic filter failure, because in my world, my filters are quite sophisticated (via Eqentia). Let me explain.

Yesterday, a tweet exchange between Paul Kedrosky and Sameer Patel caught my attention:

I replied to them “But serendipitous reading should be balanced with targeted one. FBoard has no real filters.” And Sameer Patel responded by saying that “fair point. And I agree we need that. Though sources are one form on filters too.”

My point is that we shouldn’t confuse information discovery with information filtering. The increase in social media mindshare is leading us to spend enormous amounts of times consuming serendipity-driven content, at the expense of forgetting how to properly configure powerful filters.

I believe there is a hierarchy and variety of filtering techniques to help us achieve the optimum balance between content completeness and time spent to gather the content. Here are 4 levels of filtering techniques that will yield powerful results when used together:

1) Sources. Remember RSS? It’s really about assembling the sources you’d like to read. Google Reader or Feedly excel at this approach. However, the drawback is that RSS feeds require continuous management and maintenance due to the dynamic nature of changes within these feeds.

2) People. Twitter epitomizes the people-filter paradigm. I’ve already critiqued the approach of relying solely on following people for making us “lazy” due to its ease of use. Social readers, including FlipBoard fit in this category because they rely on your network’s followers social gestures to populate various content streams.

Up to here, we were into “general-purpose” aggregation and reading where personalization is with a “small p”. The next 2 items enable “special-purpose” filtering via targeted content harvesting.

3) Keywords. Keyword filtering can range from single keywords, to long Boolean expressions, to text-mining that disambiguates meanings, and corner every nook and cranny around a particular topic. The Eqentia platform relies on such sophisticated keyword filtering that define a particular context from which content is extracted.

4) Triggers. The plot thickens. Triggers are actionable pieces of content that are the results of specific analysis that is unique to your needs. For e.g. some of the business triggers that we use at Eqentia for our company portals include Business Expansions, New Products, Partnerships, People on the Move, etc (see IBM’s portal). Triggers represent the highest levels of value because there lead to actionable recommendations (at the expense of analysis and extra processing, of course).

The reality is that you need the combination of all 4. For example, within any Eqentia custom portal, the user configures a) which sources to follow (as opened sources with no filters), b) the set of keywords (via sophisticated text mining formulas), 3) people (via Twitter Lists or users), and 4) business triggers to watch for (derived via a special analysis of the content).

There is an added element variable,– and it’s a Learning component. We, as users are just the operators. Algorithms and natural language processing can learn from our actions or results-to-date to improve, suggest or refine what we see next.

And other signals can serve to amplify or hide content. These signals include the ones coming from social media (attention gestures), relevancy matching (based on actual text analysis), and other authority-based metrics (number of clicks, source influence).

Filters are everywhere. Some will give you knowledge sponging  ad nauseam and others will lead you to relevancy and actions. Be sure to have the right mix of filters before your reading times are up.

Related articles

• Do We Have Too Many Filters, Or Not Enough? (gigaom.com)
• Information overload and failure filter are false problems (thewayoftheweb.net)
• Making sense of and filtering information overload (rossdawsonblog.com)