Knowledge Management

Big data: opportunity or threat?

‘Big data’ wants to choke your organization. Don’t let it happen.

What is big data?

The McKinsey Global Institute report Big data:  The next frontier for innovation, competition, and productivity (May 2011) is a relatively well-informed and hype-free description of the opportunities presented by Big Data.  They define Big Data as “datasets whose size is beyond the ability of typical database software to capture, store, manage, and analyze.”

It’s important to note that they define it in terms of a functional capability (the last five words of their definition), rather than any absolute in terms of dataset size.  If we can manage and analyze it, by their definition it’s no longer ‘big’.

I’d argue that McKinsey’s definition does not go far enough, and that we should actually be concerned with doing something value-productive with the resulting analysis — an argument I’ll develop further below.

How big data works

What we used to call ‘reality’ has now morphed from ‘analog reality’ to ‘digital reality’.  Nearly everything has digital sensors on it — all this data goes into one big pool — it’s similar to what we used to call data warehousing, except this time on steroids.

Then various metrics are captured and analyzed for correlation.  When certain factors move together with a certain level of statistical reliability, they are said to be ‘correlated’.  It’s tempting to assume they are also related in some causal way — but this is often premature at best.  What is needed in addition — and it’s a big methodological step — is a mechanism of causation.

Big Data is essentially a data-up approach.  The economic business case for it rests on the fact that data collection and storage have become relatively inexpensive – often wholly automated through collection sensors, telecommunications, servers, and so on.  The rationale continues that if it incurs little cost to collect, it’s best to collect it just in case — even if the reason for doing so is initially unclear or non-existent.

The hidden fallacy

This rationale behind Big Data sounds reasonable enough — but there’s a catch, a big one.  The central problem with this logic is that it neglects to price in (i.e., it treats as ‘free’) the most costly and rare resource in the ‘knowledge value chain’ — the human attention and processing needed to convert the analyzed information into decisions and actions.  Without that (so the KVC model says) there is little possibility of creating value, however that’s defined by your organization.

If the human processing element were built into the equation, the ROI would look much different.  Then it would become clear that just because you CAN gather some data, does not necessarily mean you SHOULD from a cost-effectiveness point of view.

Read the rest of this entry »

Information overload: an urban myth?

I just listened to a fascinating webinar in which five authors recounted their experiences, both personal and professional, with information overload.  One of the speakers, Jonathan Spira, reports that he has measured this phenomenon, and that it costs the US economy over $1 trillion per year!

Shifting the blame

But in naming the phenomenon ‘information overload’, it seems to place the blame on the information—not on ourselves, where it belongs.  People bemoan the distractions offered by ubiquitous information devices like smart phones and tablets—similar to the complaints that were made when the telegraph and the printed book were introduced!

This seems to us like blaming obesity on food.  “Gee, there’s so much food out there, if I ate it all, I’d get really fat.”  (Yes, you would…and with an obesity rate running over 30% in many US states, some people appear to be trying to do just that.)  If we were to talk about obesity as a food overload, it would sound pretty silly.  Though our general abundance is certainly an enabling factor, most people realize that you have to eat intelligently and selectively to stay healthy.

Maybe what we’re all experiencing is better described as collective attention deficit, or of being focus-challenged.

The information metabolism

all-you-can-eat_25Organizations have this problem too, and typically don’t fare much better.  In my article “The Information Metabolism” (Competitive Intelligence Review, Fall 1995), I compared the intake and processing of information to the intake and processing of food—eating and digestion. Though my tone was whimsical, I was only half-kidding.   I believe they are closely analogous, as my article described:

Some organizations are ‘information starved’…Others are ‘fat’ with information—they acquire it, but can’t use it effectively to create value.  Still other organizations ‘binge’ on information—they get lots of it at certain times (like strategic planning season), but not enough the rest of the time.

This was written at the dawn of the Internet Age, and the situation has accelerated dramatically since then.  I’ve seen this up close in companies, and it can be quite distressing.  Some are literally awash in so much data that it erodes their ability to process and use it effectively to manage their business.

Read the rest of this entry »

Rebalance your knowledge portfolio

If you have any experience with investing, you know about rebalancing your portfolio.  Every so often—at the end of every year, say—you need to reassess your investments.  Some may have grown, such that you’re too heavily invested in a particular stock or sector in the economy.  In other areas, you may find that you have less invested than you would like, given the prospects for another higher-potential sector.

Why do we need to do this?  Because the world changes, and the asset mix may no longer meet our needs.  I may feel that the economy is risky, and I’d rather be more invested in safer things like bonds than stocks.

In addition to the ebb and flow of securities prices, my needs may also change.  I may need to take out money for my child’s education, or for other unexpected expenses.  I may feel that I’d rather have more insurance.

The knowledge portfolio

It’s much the same way for organizational information and research assets.  They are financial investments—not costs—and each one brings to the organization a return in the form of optimized decisions and business results.  I find it helpful to think of all the research, information, and intelligence in an organization as its ‘knowledge portfolio’, with each research initiative, report, and staff position representing a specific asset in that portfolio.

Read the rest of this entry »

The knowledge archipelago

We’re living amidst a paradox in our relationship to information.  A recent University of California study finds that the average American consumes information for nearly 12 hours a day, sucking down about 34 gigabytes during that time.  Yet, we are continually surprised by stock market crashes, assets bubbles…and who knows what next?

We suffer from a quantity-relevance gap with regard to information.  Simply put, the voluminous information we have is not always the information we most need.   As a result, “data overload” or “information anxiety” are maladies that many of us experience often.

Part of the problem is the escalating proliferation of information sources.  This started back in the last decades of the 20th century when the number of magazines and TV channels began growing exponentially.  And the subsequent growth of the Internet in general, and of social media in particular, makes that pale by comparison.

You might explain this by saying that a household—being the smallest type of organization—is especially inept when it comes to information usage, and assume that larger organizations would have figured it all out by now.  But in my experience they suffer from the same kinds of problems, often compounded by being up-scaled and highly complex.

Large organization C-level decision makers typically report that they have to make bigger decisions, in shorter time frames—and have less than they would prefer of the information to do so rationally.

Not enough information?  When companies spend on average more than 5% of revenues on information??  How is this possible???

Read the rest of this entry »

Competing in the knowledge economy

Happy New Year!  Loyal readers will notice the new look, feel, and features of this site.  I want to acknowledge the talents and hard work of our developer, Tyler Gore, in making this all happen.

We’re also re-positioned the site.  It started life as a commercial site for the Knowledge Value Chain® and related activities.  While the KVC remains our “sponsor”, we feel that there is a higher purpose to be served.  Namely, observing and helping to foster an understanding of the massive economic shifts that we are now experiencing worldwide…in ways that we hope are informed, insightful, interesting—and most of all, useful to our readers.

Our economy has changed

What is the “knowledge economy”?  The term dates back over forty years to 1968, when management thinker Peter Drucker described it in detail in the chapter of the same name in his book The Age of Discontinuity.  In his words,  “From an economy of goods, which America was as recently as World War II, we have changed into a knowledge economy…The productivity of knowledge has already become the key to productivity, competitive strength, and economic achievement.

Drucker goes on to say that, where the center of the American work force had until that time been the assembly-line factory worker, “Today the center is the knowledge worker, the man or woman who applies to productive work ideas, concepts, and information rather than manual skill or brawn” [my emphasis].  That general description probably applies to most of you reading these words.

Read the rest of this entry »

Knowledge management ground zero

We recently had to deal directly with a Very Large Phone Company, on whom most of the businesses in my Manhattan neighborhood ultimately rely for both their telephone and Internet connections.  In the process of getting a new phone system installed after our recent office move, my staff and I witnessed some of the most primitive “knowledge” behavior I’ve ever seen.  We noticed that, at least once a week during our ten-week (!) wait for service, a new team would arrive to scout the connections entering our new building on the street level.  Rarely the same team twice.  And each time, the team seemed to be asking similar questions about where wires came from, where they ran, and so on.  And each time (you guessed it) their report that there were not enough lines coming into the building was oddly familiar.

Our building is a 100-year-old former medical equipment warehouse in the High Line district on the Hudson River just south of the Javits Center—now a thriving art gallery district, and soon to become a thriving business district.  The building was not originally built for telephones.  As a result, connection boxes are in odd places, wires run where you’d not expect—you really have to know what you’re doing.

Read the rest of this entry »

The digital half-life

I had an experience recently that reminded me how fragile the Knowledge Value Chain really is.  A client had asked for information on the quality of intelligence, a topic that I’ve addressed in many talks over the years.  I looked through dozens of PowerPoint slide shows from these talks, and was printing the slides that were most relevant to his question.

Then, when looking though a file created about 12 years ago, I got an error message that the file was incompatible, and could not be opened.  I tried another file of the same vintage, and got the same message.

After some research on one of Microsoft’s excellent support sites, I’ve come to the conclusion that the software I’m using (PowerPoint 2007, which I find superior in all other respects to its predecessor) will not open the file I’m trying to access.

Read the rest of this entry »