Bad Night for Big Data

I have a nightmarish pet scenario that as we as a society gain non-stop access to ever-increasing data, there is a risk that we actually get progressively dumber — as we lose the ability to process and analysis that data sufficiently.

My idea got a workout this week during election night when the polling industry, most of whom had predicted a single or even double percentage point Clinton victory, got it monumentally wrong.

When we hear on TV every ten minutes about how Watson is curing cancer, among other breathless hype about Big Data, an error of this stunning magnitude seems at first paradoxical.

But the more you think about it, the more it makes a perverse kind of sense.

“Dewey Defeats Truman”

Embarrassing election errors are nothing new — witness the iconic photo of President-elect Truman gleefully displaying the newspaper headline “Dewey Defeats Truman” the day after the 1948 election.dewey-defeats-truman

People claimed then that the error was due to a combination of slow reporting and the print-era need to prepare headlines hours in advance of publication.

What IS new is that polls are now easier and cheaper to field, and as a natural consequence there is a proliferation of them. And, as they are invariably deemed newsworthy, they feed the hungry news-cycle monster. They generate eyeballs and click-bait — and they’re fun, especially when your own pick is ahead.

Especially toward the close of this 18-month campaign, it seemed like a new poll was appearing every other day. We became so collectively absorbed in the twitching poll dashboards that we neglected the fleeting opportunity to discuss in any depth the serious challenges facing our country and our society.

Read the rest of this entry »

Value Gets Lost

“I’m stuck at the bottom of the pyramid.” “My value is unclear to people who matter.” “I’m invisible.”

In conducting “Points of Pain” exercises during TKA’s workshops and on-site clinics, too often we hear things like this from competent and hard-working knowledge producers. In study after study, roughly half of the challenges expressed by PRODUCERS of organizational knowledge or intelligence involve questions or concerns about the value they generate.

More often than not, the questions are not about producing value per se — usually producers are pretty clear and confident about that. The major gap is that their client USERS do not understand this value — and that therefore they have trouble attributing the value of knowledge back to those who originally produced it.

Our output is their input

In economic terms, any knowledge or intelligence work product, while typically the OUTPUT or end product of a knowledge or intelligence process, is subsequently the INPUT or raw material for a client’s work stream. Knowledge users take over where knowledge producers leave off — that’s one of the fundamental lessons of the KVC framework. During the handoff — the Communication step — the knowledge work product is transformed into intelligence — the basis for decisions, actions, and the production of “enterprise value” (for example, a product that brings revenues into a business).

A KVC Clinic client recently pointed out to us that the KVC triangle graphic makes it appear as if value is only produced by people and processes at the top. This was a fundamental misunderstanding of the model — for which (of course) I take full responsibility.  And hereby try to correct, please read on…

Triangle and trapezoid

The Enterprise Value (EV) Triangle

The Enterprise Value (EV) Triangle

What we mean by the word Value in the “little triangle” at the top of the KVC triangle is more properly specified as “Enterprise Value” (EV). Value is produced at each one of the seven steps in the KVC process (see the KVC Handbook p. 54). But value is only realized — i.e., made manifest and measurable in terms of revenues or other organizational outcomes or results — at the top.

Using one of my favorite analogies, the people who pick the grapes ultimately get paid by the people who buy the wine — but there is a value chain of activities that separate these two economic events in time and space.

The Knowledge Value (KV) Trapezoid

The Knowledge Value (KV) Trapezoid

With knowledge services, the problem is that the production of “Knowledge Value” (KV) — what another client called “the trapezoid” below Communication — is often separated from the production of EV, in two respects: (1) separated in time, and (2) separated in organizational location.

Read the rest of this entry »

More News from the Dark Side

I pay close attention to feedback I receive on the KVC and other analytic frameworks we are developing.  Many times I make revisions based on this feedback — that’s why the KVC Handbook is now on its fourth major edition.

One of the things I’ve heard is that the KVC model is too idealistic.  Even as I confess to being idealistic by nature, I think that’s a fair criticism.  And thanks to your feedback I use this blog (and my Knowledge Clinics) to address things not in the current edition of the book.

In the private sector, examples of damaging deviations from the ideal are as easy to spot as this morning’s Wall Street Journal.  Last month, for example, I outlined the issue of what happens when the Knowledge Value Chain is broken by chance — or corrupted by intention.

Intelligence in war

This month we examine a case that has been in play for a while, from public affairs in the US.  (Though even our readers in South Africa and elsewhere should take note — things like this could happen there too!)

In general the issue is the reliability of intelligence in an active war theater — here the ongoing actions against ISIS.  Does this sound familiar?  It should — read my earlier post about General Michael Flynn’s criticism and subsequent reshaping of the intelligence effort in Afghanistan.

And those of you who (like me) are baby boomers will remember this issue as it played out in Viet Nam.

Read the rest of this entry »

The Value of Knowledge Makes Headline News

Information has its greatest value when it is most available to, and accessible by, people for immediate use in understanding their world. I not only believe this, I put this insight to work in my consulting and teaching.

To implement this, I often use stories from the headlines to illustrate my key points. There are so many examples illustrating the KVC in the news that I am confident that I can pick up a Wall Street Journal at random and find a real-world illustration of a key point.

I call this technique a “flash case” — since it has the teaching value of a standard business school case — but it has the key advantages that (1) it can be developed quickly and (2) it evolves over time as the actual events play out.

The Deutsche Bank Case

For example, I recently used the warning letters from the NY Federal Reserve Bank to Deutsche Bank (DB) about deficiencies their capital requirements reporting process. Yes, all that detailed, boring, low-level stuff — that can gut the fortunes of enterprises heretofore thought unassailable.

Deutsche Bank logoGraduate students in my audience at Columbia University were able to identify each aspect of the knowledge-value relationship in the case. Much of the discussion focused on this pivotal issue: was this a technology shortfall, or rather a systemic problem in corporate culture originating at the top? More the latter than the former was the class consensus — a view that has been largely borne out by subsequent events.

Around the same time as the capital reporting issues, DB was involved in the LIBOR-rigging scandal, in which several huge banks were found to have essentially fabricated data used to set key rates in the world financial markets. In April 2015 the bank was fined $2.5 billion by US and British authorities for its role in the scandal — more than any other single institution.

These and related issues led to a top-management shakeup at the bank in June 2015. DB’s stock currently sells for 1/3 of what it sold for at the beginning of 2014, and the cost of insuring the bank’s debt has risen significantly — a clear signal that the once-dominant institution is now considered a risky asset.

Read the rest of this entry »

The Knowledge Payout

Knowledge Management would be better off as a discipline if it leaned into the management side more, and relied a little less exclusively on the knowledge side. It’s possible to read articles or even whole books on KM and find little discussion specifying the knowledge that people use, specifically how they use it in producing value, and/or what it costs.

Productive knowledge is transitive and transactional; it is necessarily “about” something.  That something is the work that knowledge users do in producing value in the process of doing their jobs.

This reminds me of my college physics classes, where they would start discussions by positing, “Assume there is no gravity and no friction.”  This is because many basic models in mechanics work best under these idealized conditions.  But in the real world, where (regrettably) we all have to deal with both gravity and friction — the models need substantial modification before they adequately describe how things actually work.

I kept my previous post, The Research Matrix, blissfully free of such real-world considerations.  Intentionally so, since the principles of “knowledge science” — like those of physics — admittedly work best without them.

The value context

In reality, knowledge exists only in a value context — it provides benefits, but also incurs costs.  “Value” is simply the ratio of benefits to costs.  My intention here is to add that context to my previous post, and to offer you a systematic framework with which to assess knowledge benefits and costs.Value =

As our B2 list (“Nice to Know” items) grows, we still want to move things from Column B (What We Don’t Know) to Column A (What We Know) — but, in considering costs, we will surely need to make tradeoffs.  Life could be a dream if time and budgets were in infinite supply, but they are not.  So we need a “value of knowledge” payout table that considers the benefit (to solving our problem) and the cost of each “unit” of knowledge. This will enable us to create knowledge priorities driving a research agenda — essential in our world of all-too-finite resources.

“Cost versus value” is a theme I develop in the KVC Handbook (p. 40). At its most basic, it says that while the marginal COST of a given unit of knowledge is (more or less) fixed, its marginal BENEFIT is contextual based on the knowledge already possessed by the user.  Marginal benefit typically decreases over time spent in a research process, because you are adding progressively less to what you already know.

We can relate benefit and cost to each other by means of a KNOWLEDGE BENEFIT/COST RATIO (KBCR) — the “bang for your knowledge buck,” and functionally equivalent to our knowledge value or ROI.  KBCR is the function that we are seeking to maximize.

Read the rest of this entry »

The Research Matrix

The other day I received an email from “Susan”, an alumna of the Columbia IKNS program whom I had the good fortune to work with as one of my students there. Susan’s question to me was on research, which that program touches upon but doesn’t cover in great depth, and in which I have lots of experience.

Susan’s client is exploring the feasibility of entering a new industrial services market related to energy.  Susan was looking to me for guidance about how to structure and price her research proposal to them.  For my purposes here, it doesn’t matter what the industry is — though I can say it’s B2B, and not one of those industries (like technology or health care) that is often in the news.

The value question

I prefer to demystify things wherever possible, so I proposed that Susan start by determining the client’s value question: What do they want to find out, and why (i.e., what do they plan to do with the answer)?  Susan had studied the KVC model at Columbia, so she knows how value is created from knowledge by making decisions and taking actions based on that knowledge — and that Planning (“Step 0”) starts at the top.

Columns A and B

Then I went into TV detective mode. Working down the chain:  what do we know, and what do we want to know?  We essentially have two columns, A for what we know, B for what we don’t know.  A is essentially our “value-relevant knowledge inventory”, and B is our “open to buy” knowledge purchase order.

Our mission

Simply put, our mission is to move things systematically from Column B to Column A.  That’s the essence of the research agenda, represented in the diagram by the orange arrow.

Except that it’s not often that simple.  (Bet you could see that coming.)  Knowledge users/clients are people, and people are essentially and eternally curious.  So Column B starts life as a set of Minimally Required Knowledge (MRK) to solve our business problem — our “need to know” items.  (Philosophers call this paring-down approach Occam’s Razor.)

Read the rest of this entry »

Meetings Will Make You — or Break You

Most of us work in virtual meetings often, some of us almost exclusively.  People call in using Google Hangouts, Skype, GoToMeeting, WebEx, JoinMe, Free Conference, and so on.  (I’m speaking here of “virtual meetings for the rest of us,” not the high-end meeting rooms costing hundreds of thousands.)

The hybrid meeting

I’ve been part of and/or hosted lots of physical meetings, and lots of online meetings.  In a meeting I hosted the other day, I encountered a variation — the digital-analog hybrid, where some of the people are remote and some are in the room.  (I’m in New York City, where everybody passes through at one time or another.)

It’s amazing what happens around the table that people on the phone do not have access to.  Off-mic side comments, glances, smiles, tones of voice — a panoply of meta-meaning that provides richness and context, and that only those physically in the room benefit from.

This led to a misunderstanding with one of my three co-presenting colleagues who was not present in the room.  I should mention that presence or absence had nothing to do with how important each person was to the meeting, nor how important the meeting was to each of them.  It was based simply on their availability to come to NYC at that time.  (Some were connected from as far away as Europe.)

The problem

For my purpose here, it’s not so important what exactly transpired.  Let’s just say it was a miscue deriving from an agenda item that was modified at show-time by consensus of those physically around the table — but not made clear — and here’s where as meeting leader I fell short — to those calling in. In other words, it was a problem enabled by the hybrid nature of the meeting.

What I’d like to share with you — because many of you may experience this too, with regard to meetings, or other conflicts with peers — is how we resolved it.

Read the rest of this entry »

Forward to the Past

I used our recent office relocation to review some files I had not visited in a while. Though an arduous undertaking, it provided some surprising rewards. Among other things, I came across one of my first major projects, done with the great firm of Peat, Marwick, Mitchell — which soon after became KPMG.

Screen Shot 2016-02-10 at 1.41.56 PM

Out client was the Department of Taxation and Finance for New York State. I worked under a business economist, Don Welsch — a brilliant and fun guy, a real visionary.

My job was to develop a model for the economy of New York State, segmented into more than 100 sectors. The State wanted to have a computer model ready to go so they could change sales tax rates (read:  raise taxes) in selective categories if they needed to rapidly plug a revenue gap.

I had in effect become the firm’s reigning guru in sales tax during a previous engagement, and before that had studied data analysis and linear modeling at Yale. This was a unique opportunity to leverage both cognitive streams.

Things were different then

It hardly needs saying that many aspects of this assignment were much different then than they would be today. It was the Digital Dark Ages! (Though of course we didn’t know that at the time. We were just trying to get our work done in the best way possible.) The Internet was still a dozen years in the future. Personal computers were just barely on the horizon, and nowhere to be seen even at well-funded firms like KPMG.

Don had a relationship with Chase Econometrics, a timesharing service that we used for time-series data by dialing in on a TI Silent 700 terminal through an acoustic coupler, into which we plugged a landline phone. Results came back to us dot-matrix printed on proprietary thermal paper at a blazing 300 characters per second.Screen Shot 2016-02-10 at 1.53.17 PM

That was a quick sprint through the infotech graveyard — but, for its time, our process was pretty high-tech. The Silent 700 was one of the first portable terminals to achieve major commercial penetration, so we could work from an office (ours or those of our client), rather than have to go to a data center. Many of our consulting colleagues regarded us as futuristic space cadets.

Read the rest of this entry »

  • Latest Posts

  • Topics

  • Archive

    • collapse2017 (2)
    • expand2016 (8)
    • expand2015 (7)
    • expand2014 (10)
    • expand2013 (12)
    • expand2012 (9)
    • expand2011 (5)
    • expand2010 (9)
    • expand2009 (9)
    • expand2008 (11)
    • expand2007 (13)
  • About this site

    COMPETING IN THE KNOWLEDGE ECONOMY is written by Timothy Powell, an independent researcher and consultant in knowledge strategy. Tim is president of The Knowledge Agency® (TKA) and serves on the faculty of Columbia University's Information and Knowledge Strategy (IKNS) graduate program.


    "During my more than three decades in business, I have served more than 100 organizations, ranging from Fortune 500s to government agencies to start-ups. I document my observations here with the intention that they may help you achieve your goals, both professional and personal.

    "These are my opinions, offered for your information only. They are not intended to substitute for professional advice."


    We typically publish monthly on or about the 15th of each month, subject to our client workload. Use the RSS feed links below to subscribe to posts and/or comments. Better yet, follow us on Twitter @twpowell to be notified of new posts and related developments.

    Thanks for reading! Please mention us to others and add your non-spam comments and suggestions -- we value your input.


    COMPETING IN THE KNOWLEDGE ECONOMY is sponsored by the Knowledge Value Chain® (KVC), a methodology that increases the value and ROI of Data, Information, Knowledge, and Intelligence.

    The contents herein are original, except where otherwise noted. All original contents are Copyright © TW Powell Co. All rights reserved.

    All KVC trademarks, trade names, designs, processes, manuals, and related materials are owned and deployed worldwide exclusively by The Knowledge Agency®. Reg. U.S. Pat. & TM Off.


    E SCIENTIA COPIA. Knowledge is the Engine of Value.