Bad Night for Big Data

I have a nightmarish pet scenario that as we as a society gain non-stop access to ever-increasing data, there is a risk that we actually get progressively dumber — as we lose the ability to process and analysis that data sufficiently.

My idea got a workout this week during election night when the polling industry, most of whom had predicted a single or even double percentage point Clinton victory, got it monumentally wrong.

When we hear on TV every ten minutes about how Watson is curing cancer, among other breathless hype about Big Data, an error of this stunning magnitude seems at first paradoxical.

But the more you think about it, the more it makes a perverse kind of sense.

“Dewey Defeats Truman”

Embarrassing election errors are nothing new — witness the iconic photo of President-elect Truman gleefully displaying the newspaper headline “Dewey Defeats Truman” the day after the 1948 election.dewey-defeats-truman

People claimed then that the error was due to a combination of slow reporting and the print-era need to prepare headlines hours in advance of publication.

What IS new is that polls are now easier and cheaper to field, and as a natural consequence there is a proliferation of them. And, as they are invariably deemed newsworthy, they feed the hungry news-cycle monster. They generate eyeballs and click-bait — and they’re fun, especially when your own pick is ahead.

Especially toward the close of this 18-month campaign, it seemed like a new poll was appearing every other day. We became so collectively absorbed in the twitching poll dashboards that we neglected the fleeting opportunity to discuss in any depth the serious challenges facing our country and our society.

Let’s figure this out

I’m confident that over the coming weeks we will see a vast, rolling post-mortem on how things went so wrong — discuss amongst yourselves — and please tell us what you came up with. It’s way too important not to figure this out.

Some of the early hypothesis include:

  • SAMPLING ERROR. The sample selection was biased by cord-cutting, the tectonic rolling shift in the US from phone landlines to wireless.
  • THE NEWS TEAM BUBBLE. Most of the major media are based in cities on the east and west coasts (New York, Washington, Atlanta, LA). People talking to like-minded people creates an echo-chamber effect, where differing perspectives tend to remain largely unheard, much less tolerated.
  • THE CHAOS EFFECT. Voter behavior is complex, and can be influenced by small, apparently non-related events — like public leaks of hacked email threads. One commentator compared it to weather forecasting in this regard, evoking chaos theory.

Not so fast, would be my retort on this last one. The weather is inanimate, and will happen regardless of what we forecast about it. In elections, by contrast, forecasts are used to allocate resources in real time. A forecast that Wisconsin would vote Democratic caused the Clinton campaign not to show up there even once after the convention — leading them to lose by a small margin. This gives new meaning to the term false positive.

As I write this, I note that intelligence expert John McGonagle, who is also my colleague and friend, has blogged about this.

Data is not deterministic

The New York Times’ Upshot column — one of my must-reads these days — was a bellwether of this. Nate Cohn’s September 20 headline says it all: “We Gave Four Good Pollsters the Same Raw Data. They Had Four Different Results.” These results, which included the Upshot’s own analysis, projected two larger and two smaller Clinton wins and one small Trump win — all from the identical data set of Florida voters.

Trump won by one percent in Florida, and only one expert team (Stanford/ Columbia/ Microsoft) called that right. Cohn attributes this primarily to two factors: (1) the team’s use of voting history, rather than stated intention to vote, to indicate likelihood of voting — a key factor when you realize only a little more than half of registered voters actually voted, and (2) their use of statistical modeling in weighting voter characteristics from the interviewed sample.

The key point here is — data by itself is not deterministic. Data does not “decide” anything by itself — the processing and analysis that follow are essential elements of the “value” equation — in this case, measured by whether or not you got it right.

Data blindness

In my recurring nightmare, we get dumber as — and even because — we get more data. In KVC terms, we cycle endlessly around the bottom levels of the chain without gaining enough momentum to leap up a level or two and see what it all means. I have termed this “data blindness,” a variation of which I think describes the Election Projection Debacle of 2016.

Of course, there is much hand-wringing, even some falling on swords (albeit soft ones).  Forecaster Dr. Sam Wong of Princeton went on national TV to fulfill his promise that he would eat a bug if his forecasts were wrong. Other forecasters are busy back-spinning their stories to explain how they actually warned people (somewhere in the fine print) that they could be wrong.

We should neither stop forecasting nor exercise the option, however tempting, of dismissing the entire forecasting industry out-of-hand. We should commit to doing much better — at polling, at critically analyzing the results, and at communicating what those polls signify — and what they do not.

Value Gets Lost

“I’m stuck at the bottom of the pyramid.” “My value is unclear to people who matter.” “I’m invisible.”

In conducting “Points of Pain” exercises during TKA’s workshops and on-site clinics, too often we hear things like these verbatim quotes. In a typical group, about half of the challenges expressed by organizational PRODUCERS of knowledge or intelligence involve questions or concerns about the value they produce.

More often than not, the challenges are not primarily about producing value — usually producers are pretty clear and confident about that. The major gap is that their USERS do not understand this value — and that therefore they have trouble attributing the value of knowledge back to those who originally produced it.

Our output is their input

Any knowledge or intelligence work product, while typically the OUTPUT or end product of a knowledge or intelligence process, is subsequently the INPUT or raw material for a client’s work stream. Our clients take over where we as producers leave off — that’s what the KVC model illustrates. During the handoff — the Communication phase — the knowledge work product is transformed into the basis for decisions, actions, and the production of “enterprise value” — a product that brings revenues into a business, for example.

A Clinic client recently pointed out that the KVC triangle graphic makes it appear as if value is only produced by people and process at the top. This is a fundamental misunderstanding of the model, and a very logical one — for which (of course) I take full responsibility.

Triangle and trapezoid

The Enterprise Value (EV) Triangle

The Enterprise Value (EV) Triangle

What we mean by the word Value in the “little triangle” at the top of the KVC triangle is more properly specified as “Enterprise Value” (EV). Value is produced at each one of the seven steps in the KVC process (see the KVC Handbook p. 54). But value is only realized — i.e., made manifest and measurable in terms of revenues or other organizational outcomes or results — at the top.

Using my favorite analogy, the people who pick the grapes ultimately get paid by the people who buy the wine — but there is a value chain of activities that separate these two events in time and space.

The Knowledge Value (KV) Trapezoid

The Knowledge Value (KV) Trapezoid

The problem is that the production of “Knowledge Value” (KV) — what another client calls “the trapezoid” below Communication — is often removed from the production of EV, in two respects: (1) it is removed in time, and (2) it is removed in organizational location.

Read the rest of this entry »

More News from the Dark Side

I pay close attention to feedback I receive on the KVC and other analytic frameworks we are developing.  Many times I make revisions based on this feedback — that’s why the KVC Handbook is now on its fourth major edition.

One of the things I’ve heard is that the KVC model is too idealistic.  Even as I confess to being idealistic by nature, I think that’s a fair criticism.  And thanks to your feedback I use this blog (and my Knowledge Clinics) to address things not in the current edition of the book.

In the private sector, examples of damaging deviations from the ideal are as easy to spot as this morning’s Wall Street Journal.  Last month, for example, I outlined the issue of what happens when the Knowledge Value Chain is broken by chance — or corrupted by intention.

Intelligence in war

This month we examine a case that has been in play for a while, from public affairs in the US.  (Though even our readers in South Africa and elsewhere should take note — things like this could happen there too!)

In general the issue is the reliability of intelligence in an active war theater — here the ongoing actions against ISIS.  Does this sound familiar?  It should — read my earlier post about General Michael Flynn’s criticism and subsequent reshaping of the intelligence effort in Afghanistan.

And those of you who (like me) are baby boomers will remember this issue as it played out in Viet Nam.

Read the rest of this entry »

The Value of Knowledge Makes Headline News

Information has its greatest value when it is most available to, and accessible by, people for immediate use in understanding their world. I not only believe this, I put this insight to work in my consulting and teaching.

To implement this, I often use stories from the headlines to illustrate my key points. There are so many examples illustrating the KVC in the news that I am confident that I can pick up a Wall Street Journal at random and find a real-world illustration of a key point.

I call this technique a “flash case” — since it has the teaching value of a standard business school case — but it has the key advantages that (1) it can be developed quickly and (2) it evolves over time as the actual events play out.

The Deutsche Bank Case

For example, I recently used the warning letters from the NY Federal Reserve Bank to Deutsche Bank (DB) about deficiencies their capital requirements reporting process. Yes, all that detailed, boring, low-level stuff — that can gut the fortunes of enterprises heretofore thought unassailable.

Deutsche Bank logoGraduate students in my audience at Columbia University were able to identify each aspect of the knowledge-value relationship in the case. Much of the discussion focused on this pivotal issue: was this a technology shortfall, or rather a systemic problem in corporate culture originating at the top? More the latter than the former was the class consensus — a view that has been largely borne out by subsequent events.

Around the same time as the capital reporting issues, DB was involved in the LIBOR-rigging scandal, in which several huge banks were found to have essentially fabricated data used to set key rates in the world financial markets. In April 2015 the bank was fined $2.5 billion by US and British authorities for its role in the scandal — more than any other single institution.

These and related issues led to a top-management shakeup at the bank in June 2015. DB’s stock currently sells for 1/3 of what it sold for at the beginning of 2014, and the cost of insuring the bank’s debt has risen significantly — a clear signal that the once-dominant institution is now considered a risky asset.

Read the rest of this entry »

The Knowledge Payout

Knowledge Management would be better off as a discipline if it leaned into the management side more, and relied a little less exclusively on the knowledge side. It’s possible to read articles or even whole books on KM and find little discussion specifying the knowledge that people use, specifically how they use it in producing value, and/or what it costs.

Productive knowledge is transitive and transactional; it is necessarily “about” something.  That something is the work that knowledge users do in producing value in the process of doing their jobs.

This reminds me of my college physics classes, where they would start discussions by positing, “Assume there is no gravity and no friction.”  This is because many basic models in mechanics work best under these idealized conditions.  But in the real world, where (regrettably) we all have to deal with both gravity and friction — the models need substantial modification before they adequately describe how things actually work.

I kept my previous post, The Research Matrix, blissfully free of such real-world considerations.  Intentionally so, since the principles of “knowledge science” — like those of physics — admittedly work best without them.

The value context

In reality, knowledge exists only in a value context — it provides benefits, but also incurs costs.  “Value” is simply the ratio of benefits to costs.  My intention here is to add that context to my previous post, and to offer you a systematic framework with which to assess knowledge benefits and costs.Value =

As our B2 list (“Nice to Know” items) grows, we still want to move things from Column B (What We Don’t Know) to Column A (What We Know) — but, in considering costs, we will surely need to make tradeoffs.  Life could be a dream if time and budgets were in infinite supply, but they are not.  So we need a “value of knowledge” payout table that considers the benefit (to solving our problem) and the cost of each “unit” of knowledge. This will enable us to create knowledge priorities driving a research agenda — essential in our world of all-too-finite resources.

“Cost versus value” is a theme I develop in the KVC Handbook (p. 40). At its most basic, it says that while the marginal COST of a given unit of knowledge is (more or less) fixed, its marginal BENEFIT is contextual based on the knowledge already possessed by the user.  Marginal benefit typically decreases over time spent in a research process, because you are adding progressively less to what you already know.

We can relate benefit and cost to each other by means of a KNOWLEDGE BENEFIT/COST RATIO (KBCR) — the “bang for your knowledge buck,” and functionally equivalent to our knowledge value or ROI.  KBCR is the function that we are seeking to maximize.

Read the rest of this entry »

The Research Matrix

The other day I received an email from “Susan”, an alumna of the Columbia IKNS program whom I had the good fortune to work with as one of my students there. Susan’s question to me was on research, which that program touches upon but doesn’t cover in great depth, and in which I have lots of experience.

Susan’s client is exploring the feasibility of entering a new industrial services market related to energy.  Susan was looking to me for guidance about how to structure and price her research proposal to them.  For my purposes here, it doesn’t matter what the industry is — though I can say it’s B2B, and not one of those industries (like technology or health care) that is often in the news.

The value question

I prefer to demystify things wherever possible, so I proposed that Susan start by determining the client’s value question: What do they want to find out, and why (i.e., what do they plan to do with the answer)?  Susan had studied the KVC model at Columbia, so she knows how value is created from knowledge by making decisions and taking actions based on that knowledge — and that Planning (“Step 0”) starts at the top.

Columns A and B

Then I went into TV detective mode. Working down the chain:  what do we know, and what do we want to know?  We essentially have two columns, A for what we know, B for what we don’t know.  A is essentially our “value-relevant knowledge inventory”, and B is our “open to buy” knowledge purchase order.

Our mission

Simply put, our mission is to move things systematically from Column B to Column A.  That’s the essence of the research agenda, represented in the diagram by the orange arrow.

Except that it’s not often that simple.  (Bet you could see that coming.)  Knowledge users/clients are people, and people are essentially and eternally curious.  So Column B starts life as a set of Minimally Required Knowledge (MRK) to solve our business problem — our “need to know” items.  (Philosophers call this paring-down approach Occam’s Razor.)

Read the rest of this entry »

Meetings Will Make You — or Break You

Most of us work in virtual meetings often, some of us almost exclusively.  People call in using Google Hangouts, Skype, GoToMeeting, WebEx, JoinMe, Free Conference, and so on.  (I’m speaking here of “virtual meetings for the rest of us,” not the high-end meeting rooms costing hundreds of thousands.)

The hybrid meeting

I’ve been part of and/or hosted lots of physical meetings, and lots of online meetings.  In a meeting I hosted the other day, I encountered a variation — the digital-analog hybrid, where some of the people are remote and some are in the room.  (I’m in New York City, where everybody passes through at one time or another.)

It’s amazing what happens around the table that people on the phone do not have access to.  Off-mic side comments, glances, smiles, tones of voice — a panoply of meta-meaning that provides richness and context, and that only those physically in the room benefit from.

This led to a misunderstanding with one of my three co-presenting colleagues who was not present in the room.  I should mention that presence or absence had nothing to do with how important each person was to the meeting, nor how important the meeting was to each of them.  It was based simply on their availability to come to NYC at that time.  (Some were connected from as far away as Europe.)

The problem

For my purpose here, it’s not so important what exactly transpired.  Let’s just say it was a miscue deriving from an agenda item that was modified at show-time by consensus of those physically around the table — but not made clear — and here’s where as meeting leader I fell short — to those calling in. In other words, it was a problem enabled by the hybrid nature of the meeting.

What I’d like to share with you — because many of you may experience this too, with regard to meetings, or other conflicts with peers — is how we resolved it.

Read the rest of this entry »

Forward to the Past

I used our recent office relocation to review some files I had not visited in a while. Though an arduous undertaking, it provided some surprising rewards. Among other things, I came across one of my first major projects, done with the great firm of Peat, Marwick, Mitchell — which soon after became KPMG.

Screen Shot 2016-02-10 at 1.41.56 PM

Out client was the Department of Taxation and Finance for New York State. I worked under a business economist, Don Welsch — a brilliant and fun guy, a real visionary.

My job was to develop a model for the economy of New York State, segmented into more than 100 sectors. The State wanted to have a computer model ready to go so they could change sales tax rates (read:  raise taxes) in selective categories if they needed to rapidly plug a revenue gap.

I had in effect become the firm’s reigning guru in sales tax during a previous engagement, and before that had studied data analysis and linear modeling at Yale. This was a unique opportunity to leverage both cognitive streams.

Things were different then

It hardly needs saying that many aspects of this assignment were much different then than they would be today. It was the Digital Dark Ages! (Though of course we didn’t know that at the time. We were just trying to get our work done in the best way possible.) The Internet was still a dozen years in the future. Personal computers were just barely on the horizon, and nowhere to be seen even at well-funded firms like KPMG.

Don had a relationship with Chase Econometrics, a timesharing service that we used for time-series data by dialing in on a TI Silent 700 terminal through an acoustic coupler, into which we plugged a landline phone. Results came back to us dot-matrix printed on proprietary thermal paper at a blazing 300 characters per second.Screen Shot 2016-02-10 at 1.53.17 PM

That was a quick sprint through the infotech graveyard — but, for its time, our process was pretty high-tech. The Silent 700 was one of the first portable terminals to achieve major commercial penetration, so we could work from an office (ours or those of our client), rather than have to go to a data center. Many of our consulting colleagues regarded us as futuristic space cadets.

Read the rest of this entry »

Piercing the Enterprise Bubble

IMG_2415

Harkness Tower, Yale

A few weeks ago I attended a reunion at my alma mater, Yale University.  As they always do, Yale offered up some of its most articulate faculty and administrators to describe the current state of affairs at the University.

The array of talent, initiative, and innovation on display was dazzling.  By the end of the two days, many of those of us who attended college decades ago were ready to sign up for another round — things have changed that much in the interim.

The Yale bubble

One surprisingly interesting session featured current administrators and faculty commenting on the current state of the University.  One dean mentioned what she calls the Yale bubble. It seems that students expect, and routinely receive, such high levels of performance from themselves and from the institution that they experience culture shock when they step outside its boundaries.

As one current student put it, “At Yale, it can be easy to get absorbed in our work, our activities, and friends. It can be easy to surround ourselves with a nice little Yale bubble.”  She goes on to describe how she and some of her friends broke out of that bubble to raise money for a disaster relief effort after a hurricane in the Philippines.

More recent events could be interpreted as showing the downside of the Yale bubble — a potential loss of balance and perspective as to what really matters.

Corporate culture — for better and for worse

Every enterprise creates its own nexus of practices, protocols, traditions, mythologies, and values — strands that together weave the fabric we call corporate culture. When you count over 300 years of history, $24 billion in the bank, and US presidents and other world leaders among your alumni, as Yale does, it’s easier than average to pull this off.

But every enterprise builds this cultural bubble, whether intentionally or not, and whether successfully or not.  It’s an essential part of what binds people to the enterprise — and thereby to each other — in collective pursuit of some common goal.

In some cases the enterprise bubble is “bubble-istic” — fluid, transparent, and porous.  It alternately expands and shrinks to fit new circumstances.  It is welcoming and inclusive.

In other cases, the enterprise bubble is made of steel and concrete.  It is hard, inflexible, exclusionary, and restrictive.  (North Korea might fit this model, for example.)

Read the rest of this entry »

The Competitive Runway

I read the following headline recently in the Wall Street Journal:  “Consumers crave [PRODUCT], but [PRODUCERS] enjoying their best profits ever are reluctant to switch.”  (The words I’ve bolded here were specified in the article, but I’ll get to that in a minute.)

Headlines reminiscent of this have been written many times in business history.  They are often prelude to disaster in the form of self-imposed obsolescence.

runway

Regarding Kodak, for example, one might in the late 20th century have written [digital technologies] and [film manufacturers] in the respective slots.  The profitability on film was so great that Kodak persisted in making and selling it, and famously did not invest soon enough in a switch over to digital. This was a titanic strategic blunder from which the company never recovered, eventually filing for Chapter 11 bankruptcy in 2012.

Willful ignorance

It happens constantly, in all industries, that consumer preferences migrate — sometimes so slowly that it’s hard to notice — until the change has become the new normal.  It’s more noticeable in B2C industries than B2B, but it happens in the latter too.

What makes these changes especially difficult to respond to is our near-universal tendency to gloss over and ignore that which could be unpleasant — or even fatal.  Our “all good news, all the time” corporate cultures make it tempting to look the other way and hope such problems will resolve themselves before metastasizing.

On the other hand, companies that have an innovation philosophy that demands that they “Obsolete ourselves before someone else does” have the upper hand.  Intel and Jobs-era Apple were famous for thriving under such regimes of continual, relentless self-betterment.

Read the rest of this entry »

  • Latest Posts

  • Topics

  • Archive

    • collapse2016 (8)
    • expand2015 (7)
    • expand2014 (10)
    • expand2013 (12)
    • expand2012 (9)
    • expand2011 (5)
    • expand2010 (9)
    • expand2009 (9)
    • expand2008 (11)
    • expand2007 (13)
  • About this site

    COMPETING IN THE KNOWLEDGE ECONOMY is written by Timothy Powell, an independent researcher and consultant in knowledge strategy. Tim is president of The Knowledge Agency® (TKA) and serves on the faculty of Columbia University's Information and Knowledge Strategy (IKNS) graduate program.

    ===================

    "During my more than three decades in business, I've served more than 100 organizations, ranging from Fortune 500s to government agencies to start-ups. I document my observations here with the intention that they may help you achieve your goals, both professional and personal.

    "These are my opinions, offered for your information only. They are not intended to substitute for professional advice."

    ===================

    We typically publish monthly on or about the 15th of each month, subject to our client workload. Use the RSS feed links below to subscribe to posts and/or comments. Better yet, follow us on Twitter @twpowell to be notified of new posts and related developments.

    Thanks for reading! Please add your comments and suggestions, we'd like to hear from you. (But no URLs or spam, please.)

    ===================

    COMPETING IN THE KNOWLEDGE ECONOMY is sponsored by the Knowledge Value Chain® (KVC), a methodology that increases the value and ROI of Data, Information, Knowledge, and Intelligence.

    The contents herein are original, except where otherwise noted. All original contents are Copyright © TW Powell Co. All rights reserved.

    All KVC trademarks, trade names, designs, processes, manuals, and related materials are owned and deployed worldwide exclusively by The Knowledge Agency®. Reg. U.S. Pat. & TM Off.

    ===================

    E SCIENTIA COPIA. Knowledge is the Engine of Value.