about 7 years on 2010-01-30


i was honored that many people whom i truly respect took the time to read my thoughts on 2000-2010-2020 and offer insightful feedback. one prediction that a lot of people, including tim o'reilly, challenged me on was my claim that by 2020 the 'relative balance of power in the media would shift from content distribution to content creation'. tim pointed out rightly that this prediction has been made time and time again with new technologies, and tends to be proven wrong at every turn. because of this and some other recent comments flying around about the future of privacy/content/etc., i figured it would be worthwhile to explain a bit more deeply why i am willing to make the 'content creator' bet, and more broadly the framework i am currently operating under -- humor me, this is going to be my big my big a priori exercise for the year, so i am going to start at the very beginning with a few definitions and build up -

recognizing that this is a very long post, let me give a way the conclusion up front:

a. the value of a given bit of information's is based on its own scarcity, but its value is relatively agnostic to the value of other information(s), because information is by definition unique. ever greater leverage in the ability of individuals to harvest the value of information means that the spoils are shifting more and more towards content creators and away from information distributors.

b. entertainment's value is a function of its scarcity relative to the market of other content/supplements, but not it's own distribution/scarcity. highly efficient filtering/distribution technology (social/management structures for managing increased interactions) give content creators relatively more leverage when you get several standard deviations away from the mean, which is where we are rapidly headed.

c. ultimately, leverage moves to content creators in the next decade because while content is abundant, valuable content (both information and entertainment flavors) is scarce. the difference between information and entertainment is only the difference in types of scarcity that drive end value for content creators.

1: a given bit of content can be classified as information or entertainment, but never both, the only way to usefully bracket a definition of content is as value-able expression that can be reduced to binary, and whose value fits under one of two possible mece (mutually exclusive collectively exhaustive) sub-classes, defined by how their value changes as a function of scale -- namely information (the price of walmart's stock tomorrow) and entertainment (harry potter books).

first, i say value-able because i would argue that white-noise, or bits that literally have zero value are not content. it isn't content if someone isn't willing to pay something for it in some way, shape, or form (which is at a baseline being willing to spend the time to consume it). second, since content is value-able, it's value can be plotted vs. another measurable characteristic, like distribution. all content has a definite unit economic value signature as a function of scale where information is defined as the pattern in which value per incremental unit of distribution erodes, and entertainment is where value per incremental unit of distribution is flat or positive.

so, while you might have a composite container of contents, like a news article, where some bits are information, and some bits are entertainment, you can ultimately reduced and separate the media to its component contents (and likely in our world would likely maximize their value in so doing)...

a. information is content which has a fixed and finite value, and whose value is harvested with distribution. incremental distribution always means a loss of value. a given piece of information becomes less and less valuable the more it is shared/used, and because of that the total value of a piece of information content is finite. if i am the only one that knows walmart's stock price tomorrow at noon (with 100% certainty) that information is priceless, if everyone knows it is worthless. in this way, you can model information as a natural resource/oil. in its pure form it begins it's life as potential value, and then is converted/liquidated into real/fungible value. information has a fixed value that relative to distribution starts high (potential energy) and ends low (kinetic energy). **to argue this in the true abstract and talk about things like the value of the wall street journal, it is important to roll in that the value of all bits of information are by definition probabilistic (what the information says * how likely it is to be true) -- but i will save that for another post because it is non-core to why content creators will get more valuable.

b. entertainment is content whose value stays constant, and might even grow with distribution. unlike information, whose value is finite and absolute, entertainment's value exists only relative to the market of other entertainment content available to me. so the dark night is equally enjoyable to me regardless of whether everyone else saw it or not, and its value might actually increase with social distribution (as i might get more and more value/enjoyment out of it the more i can discuss it with friends), but varies relative to what else is available to me in any given moment. overall, entertainment's value is theoretically unlimited relative to its own distribution. ** for a later post: a meme is a flavor of entertainment where the value of the content at scale is many multiples of the value of the content without scale.

so, back to a real world and seemingly muddled case - a news article- any grouping of bits might have components which are information and components which are entertainment... but i would expect that the trends will continue to be the disaggregation of the two into component parts because information and entertainment are at economic odds with one another. when you wrap information and entertainment into the same body, you dilute the economic power of each because your incentive set on scale conflicts with itself.

2. information: technology is giving content creators ever greater leverage, allowing them to harvest a larger precent of the total value of a new bit of information: because information has a finite value, harvesting value from information is a race to the bottom. when t=0 and a new piece of information comes into the world, its value is 100% of what the entire known universe would pay for it less the total expense of fully disseminating that information. when t=infinity and a piece of information is perfectly known by everyone everywhere, its value is 0% of what the entire known universe would pay for it. (of course, the 0% can actually sit at a negative absolute dollar amount as a function of distribution costs, in which case the information would stop spreading before everyone knows it)

for the sake of a silly example, let's play pirate: you get a map to some buried treasure 2000 miles away. to simply things you have 100% confidence that it is accurate - so we set aside the fundamentally probabilistic nature of the information).

- if you are the only one with the map, the value of the intel is: the value of the treasure - the cost of extraction (and you could probably sell it for $1 less than that).
- if everyone has the map then no one is going to buy it from you because it is worthless (like the good old joke about the economist that stumbles across the $100 in the street).
- if just a few people have the map, you could sell the map for somewhere between 100% and 0% of the initial value.

a. the total value of information has risen for many types of information, because the market has grown and therefor a given new piece of knowledge is worth more: just as a warmup (not critical for our conclusion), it is undeniable that for many (but not all) forms of information the total value of information has shifted up as the total human population, and the total size of the economy, has grown. possibly the easiest example of this is the lottery. the value of somehow knowing the winning lottery number is much higher now than it was 30 years ago, because more people now play the lottery. this absolute shift up isn't all that interesting, but it is worth noting. this is a useful illustration largely because in the lottery the value of a winning number is a pure step function with a definite expiration date.

b. the initial value of information in most cases has grown throughout history, driven by ever more leverage to take advantage of in the total market. the communicative leverage you can put to work to harvest a new given piece of information is growing rapidly, leading to a rise in the initial value of information. let's pretend that you live in chicago and happen magically to be the only one that knows which horse is going to win the kentucky derby 20 min before the race is run. 100 years ago, maybe you run to a local bookie, and make the biggest bet you can based on the cash in your pocket and the local chicago market. now, you could pick up the phone, leverage tons of fast acting lines of credit (like credit cards) and make bets all of the world. your cost of grabbing more of the total value of the information has declined precipitously, so the information is worth more to you if you are early in the diffusion curve (value - cost).

more obvious still. look at hedge funds (or, just the concept of using a lot of financial leverage). it used to be that if you knew a great stock pick, you could buy a few shares up to the cash you had, and benefit a bit... you could tell a few friends, they could benefit also. now, hedge funds can theoretically pile on as much leverage as possible, and using derivative instruments, capture way more upside per $. one person can smartly capture the whole win up to the point of moving the market.

or, back to our treasure map whose value is the value of the bootie less the cost of extraction. 2000 years ago the extraction cost was traveling half way across the world, digging it up, hiring a ton of people to help you extract it - who will likely steal the intel from you, etc.- and then of course the cost of financing/capital to get it all done.... now, just hop a jet and rent a john deere for a few hundred bucks.

in a phrase - doing things is easier and knowing things is cheaper - so knowing things other people don't know is more valuable. so, not only is market is bigger in absolute for many, but not all, types of information (which means total value is greater), but greater leverage means that individuals early in the curve have lower cost extracting more out of the total value of information... so, returns to those whom are early are even greater...

c. ... but the corollary is that information amp information's value also diffuses much more quickly, shortening the window of opportunity for high returns. adam smith actually made quite a bit of money throughout his life in the 1700 arbitraging gold prices in europe in more or less the same way over and over. the market didn't respond - the opportunity to leverage a tiny bit of information was relatively consistent for decades. now, opportunities close much much more quickly. the window to use information is generally much shorter. this is because there are generally more smart and connected people talking, working on the same problems, and cheaply pushing around bits than ever before. further, the cost of harvesting information isn't lower just for the first person to discover/create information, it is lower for everyone in a competitive marketplace. so, the area under the curve (the total value of information) is constant... and if you harvest more of it more quickly, your ability to sit later in the curve and extract value is diminished.

to really see why converting information into value is a race to the bottom... back to pirates -- 2000 years ago, if you and five other individuals got their hands on the same 100% confidence treasure map within a few months of each-other and set out at a sprint, chances are that guy number 1 isn't finished with the labor of digging up all the treasure when competitors get there... so the 5 either raise armies to try to kill their competitors (expensive) or the cooperate and share the upside. cooperating is not as good as being first alone, but being in the first group probably means you get dealt in.... now, using my john deere backhoe, if you get the map 1 day after me, you are screwed because in that one day i will have already taken 100% of the bootie and be chilling out in michanos.

d. so, information is ever more perishable, and if you have information this means your best bet is to harvest it as quickly as possible: if you have information of value, you need to use it or loose it. as the window on inefficiencies generally closes faster and faster, people have an incentive to leverage what they know at an ever faster rate. i don't sit on ideas. i see how highly i can leverage them, i leverage them, and i move on. there are a lot of interesting dynamics around this for a later post (prisoner's dilemma is a great way to describe why information diffuses so quickly)

e. thus, by 2020 content creators - those who know first - will gain power on a relative basis: new information is the only place with leverage on the value chain - if you can generate true information, your ability to leverage the initial value of that information is greatly enhanced. but because the information diffuses so quickly, the window of opportunity is much shorter for follow-on by other people who gain access to your information. the distance between first and everyone else continues to grow... the those that are 'first' generally have more and more leverage. by this argument, value resides with being first to an insight... the second place prize has been seriously diminished.** (again, please note that this totally ignores the probabilistic nature of information, which is where things like the times/professional news make a big difference).

or - put in a phrase - the cost of harvesting information once it exists has become very low, so the value is in creation, not the pipes to disseminate. -- news organizations are valuable if they are fine tuned for news gathering and fact checking (a form of second derivative information), not because they have fleets of trucks and paperboys to deliver the news.

-- the result of this is increasing inequality. the earliest people/the information creators do better, everyone else does worse -- the potentially socially bad news (if you want to take moral sides) is that leverage/information content creators winning more reduces equality. original information can be leveraged more quickly in a wider audience, so producing it is a huge deal... but the value of second place is rapidly being diminished towards zero). ** later post: argument in here for the implications on patents/ip. 18 years is far too long in our changing landscape

-- the implications of this obviously play out for all sorts of second derivatives as well (information about information, of meaningful note given my expected audience on this). this is especially critical when considering search.... again, this is a stem for later followup -- but the basic point is that these same dynamics suggest that traditional 'search' is going to get much less valuable rapidly.

3. entertainment: competitive 'filters' are bringing individuals so many standard deviations away from mean content, that content i value is getting scarce, and power is shifting to content creators:let's start by considering in a bit more detail how baseline entertainment consumption works.... unlike information, where i have relatively elastic demand (i can use leverage/computers/companies/etc. to consume as much information as is roi positive - and i am getting more consumption leverage all the time), my entertainment consumption is relatively inelastic. i have x hours and y dollars a day to spend on being entertained (reduce it either way, but let's pretend i can trade dollars for hours and hours for dollars, so we are only dealing with one dimension).

at the same time, completely forgetting the concept of 'filters' for a second (that comes later, stick with me) we as individuals each face a world of entertainment content whose value in utils/hour looks something like a normal distribution. there is a small amount of content which is very high value (is hyper rich in utils/hour), a lot of content clumped around the mean, and a bit of truly terrible content which is really low value. of course, this curve is personal... so something which might be very very high in value on my curve could be very very low on yours.

part of the reason that i believe that a normal distribution is a good general model, is because the value of entertainment content can be understood only on a relative basis (just like intelligence, or the more modern intelligences). avitar did very well in 2009-2010 -- but it was very high on the distribution curve only compared to the other movies/forms of entertainment right now. if you had released avitar in 1970, chances are it would be considered several orders of magnitude more amazing than it is today (and would have made way more money), if you release it in 2015, we might find it relatively less entertaining/utility rich.

the upshot is that in a baseline scenario, with zero filters at my disposal, an individual will tend to get the mean number of utils with my relatively fixed time/dollar entertainment wallet/budget... an individual would consume value from entertainment at the mean of all content produced. (so -- in the below i will tend to consume at the intersection of my total consumption volume (the red line and the bell curve average (blue line)--

the story of the above graph goes something like this. you are in a room with an hour. you have at your disposal a set of un-labeled books, videos, audio, video games, etc. you have no way to know which ones are good/which ones you would like or which ones are bad. on average, over enough days in the library, you will tend to consume the mean piece of content at your disposal, and will harvest the mean number of utils from your hour(s).

a. storage amp distribution costs are dropping so there is far more content at my disposal: not too long ago, at 10pm on a tuesday i had at my disposal whatever books were in my house, whatever was on one of 5 television channels, a handful of content on the internet, and a wired phone to try to reach someone who was awake. now, at 10pm on a tuesday i have at my disposal thousands of movies, hundreds of television shows, thousands of books (on kindle), basically every piece of music that has been professionally recorded, streams of content from thousands of friends, etc. there are several orders of magnitude more entertainment content at my disposal.

further, i also on average have a slightly larger time/money entertainment wallet if i live in the western world i am working on average less than i was 10 years ago... (not of big consequence, but worth noting and indicated by the red line moving up in the below graphic)

all that said, just because distribution and storage costs are down, doesn't mean that the average value of the content has changed, for now - again without filters - for every bit of new really great stuff at my disposal, there is a ton of barney in malaysian with hindi subtitles (very valuable to malaysian kids that speak hindi, but of almost zero value to me).... and as the result of storage and distribution we find that we have more content, but more of the same content from a value distribution perspective. the library is bigger, but i get the same enjoyment out of it as i did out of the smaller library, and in either case i can only consume a small fraction of the material.

b. content creation costs are dropping, driving lower average value (utility/hour) in the content produced: if you only make one movie a year, you tend to try to pick your best script and put your best director on it. when humanity starts spitting out ?thousands? of movies a year, the average drops. so, the second thing that is changing is that the average bit of entertainment content produced in the world is dropping fast. with no hurdle to production some great stuff gets made that wouldn't have otherwise been created -- but far far far more junk gets created per incremental unit.

when there were 3 channels at 10pm i was on average 33% of the time going to end up with johnny carson if i randomly switched to a channel -- with 2000 channels, i have a 1/2000 shot of randomly getting coco, and a 3/2000 shot of getting jersey shore, real housewives of nyc, or... real housewives of orange county. so, i bet money the average telegraph sent (expensive) was way way more interesting than the average tweet (cheap). just as the average message run 26 miles up to athens after the battle of marathon was way more interesting than the average telegraph.

in step 2, still forgetting filters, my average media consumption has actually lost value recently. my consumption volume is constant, but because lower production costs ultimately ends up correlating with a shift down in our average content value curve, i the content i am consuming is actually on average worse than it was in a content production constrained environment.

c. of course, no one actually consumes the mean of the basket of content available to them, filters help us find content that we will enjoy in excess to the mean. we have always had content filters. social filters have existed as long as friendships -- the are not a modern invention. people were recommending plays/books/etc. to each-other since the beginning of time helping them leverage up the value in their finite entertainment consumption wallet. the tv networks and record labels are both pools of capital/knowledge, and filters that extend all the way backwards through the creative process. tv guide, the new york times book review, zagats, basically any editorial content about entertainment is a form of a filter.

filters look like/function by basically taking your average consumption from the mean (the blue line that has shifted down), across the orange line, up to the blue dot. the 'value' of a filter can be expressed/thought of as the length of the orange vector minus the cost of consuming the review/editorial (in time/money/etc). so, a good filter can be extremely valuable in a world awash with content. against a backdrop of abundant content, filters help you consume more efficiently, and thereby get more utils/unit of consumption. filters are the only way we can escape our otherwise clear fate of consuming ever lower value entertainment content (in terms of utils) as the mean content in the market continues to drop.

d. filters are evolving rapidly: for a long time, filters were relatively simple and provided clear but modest improvements over the mean. the first filters were real world social filters (friends, families, etc)... after that traditional editorial filters evolved - tv guide, reader's digest, etc., as well as record labels and tv networks screening content in early development, and filtering it through to your eyes (while simultaneously financing it, and contributing to it) ... skip forward to search engines (algorithmic filters) and now social networks (filters combining old school friends, families, etc. + algorithmic)... technological points of discontinuity (printing, scaled algorithmic processing, social graph, etc.) have a few times throughout history meaningfully allowed you to step up the entertainment content value curve in relatively sharp jolts.

the key is that the value of a filter in absolute is the positive shift in average consumption value it provides to users... but the value of any given filter is measured relative to other filter options/costs. filters exist in a competitive marketplace within a type (newspaper vs. newspaper) but also across the whole function (social vs. search vs. newspaper)so, the value of search is not the total vector from average content to editorial options -- it is the marginal improvement that search has over editorial. the value of social is not necessarily the value of social relative to editorial, but it's value relative to search. (again, this is another stub of an argument/many points fall out here, but hopefully the general concept is captured/serves for our end goal here)...

e. filtering itself is becoming an ultra competitive marketplace, facing commoditization and declining margins: just to belabor the point... the reality looks more like the below, where the value of any filter is only the marginal improvement on the x axis of one dot (service) over all other possible filtering choices. so, search might plop down on the content curve line on certain categories and have a decided advantage for a while, but that advantage is ultimately eroded by competing services. the same relationship plays out in a derivative with the filters themselves. as more filters evolve, the relative value of any one over the others declines.

f. why 'entertainment content creation' will start winning this decade, because we can surface great (and scarce content!): filters are getting really good, and they are going to keep getting better... and this presents the ultimate rub: filters are competing each other to provide excess value so many standard deviations above the mean content, that all of a sudden truly excellent content that i want to consume (and filters want to provide me) is scarce again, which means the balance of power swings back to those that can produce truly excellent stuff.

the basic idea is that when you are just one or two standard deviations above the mean content value, filters dominate because content that hits that richness is relatively abundant and replaceable. there is far more content at the value level required than you could possibly consume, so any given content owner/producer lacks pricing power.... but, i believe that as we approach 2020, we are going to face a scenario where we are a sufficient number of degrees above average content in what is required to compete, that filters will be delivering scarce and valuable content in an effort to compete. if filters start delivering content that isn't replaceable/commodity, then all of a sudden the content owners are going to find themselves with a lot more leverage.

below i illustrate the point at which content creators/owners gain back power -- where my daily entertainment consumption wallet exceeds the average content available to me. i am looking for a better answer than a simple filter on top of commodity content can provide. i need great content and great filters, not just great filters of mediocre content.

to keep hammering on this, entertainment content creators will start winning this decade because the filtering business is becoming just as competitive as the content business itself. to create value you need to be so many standard deviations out on the curve that the content itself is just as scarce as the filtering/finding of the content... so, where once we were awash in models of filters/distribution leveraging under-valued content, we now find that to really do well, content and filtering/distribution need each-other.

- historically: content (easy to make), distribution/filters (scarce)
- 2020: content (easy to make, hard to make well). distribution (easy to make, hard to make well)

the result is that to get outsized returns on content, you have to produce and filter content many sds above the norm. where once the market had a lot of slack in it, and filters could dominate by finding marginally better entertainment content, i would argue that we now face a situation where incredible content needs incredible filtering, and visa versa to provide any sort of actual value. you need to get all the way down the curve to the extreme right, which requires both extremely good and extremely scarce content, and extremely good and extremely scarce filtering/distribution.

where does this work?

- i want to listen to yoyo ma play bach
- i want to listen to her majesty by the beatles
- i want to read stephen pinker
- i want to watch 30 rock

g. this doesn't mean that your status update is valuable: for note now and expansion later - just because power is shifting back to content creators doesn't mean the personal entertainment content you generate is all that valuable.... even if your 50 friends like the photo you took and value it several sds above the mean content they can consume, don't kid yourself - even when amassed that content isn't worth that much. your only photo of grandma is priceless, but your 1000th baby photo is mostly worthless.

h. however, all this said, facebook and google still win - just not for the reasons everyone thinks: for note now and expansion later -- if facebook and google were just filters, the above would clearly imply that they have issues on the horizon... but facebook and google are not just filters, they are content creators. i will go into this more some other time, but for now -- just consider who really is the 'owner'/'creator' of your status update. on one hand, of course you wrote it.... but in reality wen't you just an input to facebook's content creation pump? i would argue that the best brands going forwards are the ones which are creating contexts which drive people to generate content for them. facebook and google are not filters - they are content creators and owners (sorry guys, i know you don't like to think of yourselves as media companies, but i don't think it is such a dirty thing going forward, even if it was historically).

- in closing: i buy the 2020 content business as on a relative upswing information's value is based on it's own scarcity. ever greater leverage means that the spoils are shifting more and more towards content creators - entertainment's value is a function of scarcity relative to other content, but not it's own distribution. the ultra competitive filtering market pushes demand all the way out to the edge, where content is scarce and where content creators have relatively more leverage.

original swl blogposts and letters 2007-2010