Friday, December 24, 2010

Latest News Through Wikipedia - Wikipedians are the real Citizen Journalists














People have long predicted the demise of traditional news media and the rise of the citizen journalists. Various initiatives have tried to create new media outlets on the Web, Blog, and Twitter powered by creative swarms of hobby journalists - but none of them has been a breakthrough success so far. Well, it turns out that there is such a citizen Web site, venerable old Wikipedia!

In a series of earlier projects we have analyzed collaboration among Wikipedia authors when creating new Wikipedia articles, for example studying how they collaborate as COINs in different cultures (http://www.ickn.org/documents/COINS2010_Nemoto_Gloor.pdf).
In our current project we are creating a map based on Who-works-with-whom-on-Wikipedia (the "W5-map"). We build a semantic network of concepts by constructing a link between two Wikipedia articles if the same author has worked on both articles. This W5-map shows us to what kind of articles the swarm flocks to. By repeating this process for every month in 2010 we are able to see how the W5-map changes over time.

As the whole Wikipedia includes millions of article, drawing a whole map of Wikipedia in one step is too much. Instead we employed a "snowball sampling" method, which allows us to draw a partial map by selecting a start article or editor. For our first experiment, we used the article about "Wikipedia" as the starting point. We collected the top 10 editors based on the number of edits on this article, then we gathered the top 10 articles of each editor. We repeated this steps recursively up to 3 degrees of separation from the start point. Restricting this analysis to a certain period of time (e.g. one month starting Jan. 1 2010), permits us to obtain a temporal W5 map from this start point. Applying this process repeatedly we calculated 11 snapshots of one month from Jan. 2010 to Nov. 2010. Each node corresponds to an article in Wikipedia. We draw an edge between articles A and B if there are at least 2 editors who made edits both on article A and article B.

The pictures below show our results. Each map was drawn by Gephi, and the size of the article title was determined by the undirected PageRank score of the W5 network. The major topics (based on PageRank Score) for each month are shown below. Surprisingly they reflect the major news item of the month:

Jan. 2010: 2010 Haiti earthquake
Feb. 2010: 2010 Winter Olympics
Mar. 2010: 2010 Polish Air Force Tu-154 crash
Apr. 2010: Telephone (song)
May. 2010: Gaza flotilla raid
Jun. 2010: 2010 FIFA World Cup
Jul. 2010: 2010 FIFA World Cup
Aug. 2010: 2010 Israel-Lebanon border clash
Sep. 2010: 2010 Atlantic hurricane season
Oct. 2010: Copiapo mining accident
Nov. 2010: United States diplomatic cables leak

Furthermore, we can also find clusters of articles, representing a group of similar topics (e.g. a cluster on Lady Gaga or on WikiLeaks).

This means that groups of similarly minded Wikipedians tend to aggregate around a set of articles on a topic they are most interested in.


Looking at Nov. 2010, the United States diplomatic cables leak was strongly connected to WikiLeaks and Julian Assange, which makes perfect sense because both of them are part of the WikiLeaks dispute. Bombardment of Yeonpyeong had many edges from the WikiLeaks cluster while there were no edges from the 2010 Asian Games cluster, which means that Wikipedians working on the Bombardment of Yeonpyeong are interested in the diplomatic problem, not in the topics in Asia.

Our preliminary investigation suggests that looking at Wikipedia through the W5 map might be a new way to identify latest news. We find the news of the world even if we start from a neutral article such as the one about "Wikipedia". The swarm of Wikipedians seems to be a perfect group of coolhunters and citizen journalists to report latest news on politics, celebrities, and sports.

Sunday, December 19, 2010

How Much Are People Smiling in the US, Germany, and Switzerland?

Who are happier, people in Switzerland, in Germany, or in the US? To answer this question, I looked at the use of smiley’s in Twitter tweets – smileys are those emoticons used to express one’s emotions like
:) smile
:D big grin
:( sad, frown
:P sticking the tongue out, “raspberry”

My hypothesis is that the larger the fraction of happy smileys :) and :D in all tweets containing emoticons is, the happier people in this region are.

Using Condor’s Twitter collector, I collected 24 hours worth of tweets containing the smileys listed above in 6 cities in three countries: New York and Los Angeles (USA), Berlin and Hamburg (Germany) and Zurich and Berne (Switzerland). I collected all tweets inside a radius of 25 kilometers around the geocoordinates of these 6 cities returned by Google.

The table below lists the results, showing the number of people using each emoticon in each city, as well as the betweenness centrality of the emoticon in the social network of people using it.

As we can see, there are not too many people tweeting in Berne, compared to the people in New York, which makes perfect sense, considering the number of inhabitants of Berne (130,000) compared to New York’s 19 million.

I constructed the retweet network in Condor, drawing a link from person A to person B, if B retweeted A (see network picture above). The table only lists the number of people, ignoring the number of tweets per person, as I was interested in the emotional state of each person.
The picture below visualizes the results. Percentages in the pie charts for each type of smiley are based on betweenness centrality of the people using these smileys. This also accounts for the influence of somebody who for example used two different types of smileys and is being retweeted a lot.

A few things immediately stand out:

(1) The Europeans seem much happier than the Americans!
(2) Germans seem slightly happier than the Swiss, although not by much.
(3) People in Hamburg are the happiest (68% happy smileys ":)" and ":D"), followed by the people in Zurich.
(4) People in Berne have the biggest smile (30% have ":D").
(5) People in New York are the least happy (23% of ":(") with a large margin to all other cities.
(6) People in LA are the most skeptical (27% sticking their tongue out ":P").

When looking at the most active tweeter in each of the cities, it is amazing that most are young girls and artists mostly from Indonesia. For example the most emotional person in Hamburg (130 tweets) is “Bijiganja”, an Indonesian singer and “sinner”, as can be read on his profile on Myspace. The most emotional tweeter in Berne is a girl from Brazil. This means that the good mood in Switzerland and Germany might actually be imported from other regions of the World, where people traditionally are more extrovert than the somewhat reserved Germans and Swiss.

This is very different in the US. In New York, the most active emotional tweeter is a disc jockey and radio host, mostly promoting himself, while Actress and singer ciara is the most active tweeter in LA. This shows that Twitter in the US seems to be much more used as a platform for (commercial) self-promotion, although not a particularly happy one!
Let’s hope that the mood will pick up also in the US – after all there are a lot of people from Asia and Latin America here that might improve the collective mood!

Friday, November 12, 2010

Another Day of Hope (mostly), and some Fear and Worry in the US

Today I checked on the mood of the US Population through Twitter, using Twitter’s Geotagging feature. Alan Mislove from Northeastern had already found that the mood of the nation changes over the course of the day, with people having a low over lunch, and getting collectively happier in the evening, when work is over. Using our Twitter-collector-tool built into Condor, I was able to easily replicate this result.

I counted the number of retweets about “hope”, “fear”, and “worry” in the major population centers of the US, by collecting the tweets at four 2000 kilometers circles with centers at Pittburgh (North East), Atlanta (South East) Las Vegas (South West), and Boise (North West). (see picture below) I then constructed the social network between the retweeters as described in a previous blog post. The way it is calculated, it also factors in the importance of the retweeters, where a link is drawn between two people if a person retweets a post from the other person.



The picture above shows the areas I covered, as well as the fraction of retweets on hope (green), fear (blue), and worry (red) around noon. As we can see, people are more hopeful in the West around noon EST (which is still in the morning in the West) than they are on the east coast.
The picture changes four hours later. The graph below shows hopefulness (fraction of retweets on hope/fractions of retweets on fear and worry) around noon EST and around 6pm EST. Hopfulness shoots up sharply in the Northeast (NE), Southeast (SE), and Southwest (SW). It also goes up in the Northwest (NW), although much less.

Note that there are always more retweets on hope than there are on fear or worry, showing that people are basically hopeful, particularly when work is over. Let’s hope that hopefulness will also go up in the evening in the Northwest!

Wednesday, November 03, 2010

Monitoring Midterm Election Night Through Twitter Buzz

Yesterday November 2nd 2010 was midterm election day in the US. I was curious what Twitter would tell us about the mood of the voters. It was already clear that things did not look good for the Democrats. In prior work analyzing data from 2009 we had already found that monitoring posts for the occurrence of “hope”, “happy”, “fear”, and “worry” would give us a good proxy for the mood of the population, particularly if we focused on the retweeted posts. So this time I repeatedly ran our Twitter data collector in 30 minute intervals, each time collecting the 200 most retweeted Tweets containing either hope, happy, fear, or worry. The picture below shows all tweets, with the red dots depicting the tweets containing more than one of the search words.

Measuring the betweenness value (i.e. the importance of the search term) shows that popular tweeters prefer tweeting about “happy” (32%) and “hope” (30%) over the “worry” (19%) and “fear” (19%) tweets. Note that I collected precisely the same amount of tweets for each search term (24*2*200), but then constructed the social network of the tweeters based on who retweeted whom’s post. To illustrate the point, the picture below only shows the social network (without drawing the links of the tweets to the search terms – these links were used in the first picture to calculate betweenness.)

As this picture shows, the tweeters about hope and happiness (yellow and light brown) are mixed in the center, while the tweeters about fear (blue) and worry (green) keep mostly to themselves in the periphery. So even in the Tweetersphere, happy people connect, while worriers stay put at the borders of the tweeter-network.

Here is the picture analyzing the contents of all tweets I collected containing the term "fear", displaying the semantic network of the most important terms about fear:

As the picture shows, Democrats are on the losing side, Republicans won, but the term "republican" is close to the term "jobless", so fear about continuing joblessness was the main cause for their win. Harry Reid’s win is also predicted by Twitter - the terms "Reid" and "Nevada" are connected to "won".

Now the picture with worry:


It shows Republicans worrying about a potential victory of Joe Sestak in Pennsylvania (which did not happen), Tea party members worrying about the loyalty of John Boehner, moms happy they don’t have to worry about their health care thanks to Barack Obama, and somebody worrying about loosing his Facebook account.

And here is the term view for “hope”:

It seems Justin Bieber and Harry Reid from Nevada share the stage of hope, with some tweets about the elections in California thrown in. Adam Lambert’s Halloween discussion of his costume is picked up by his fans. Barack Obama still draws lots of emotions and inspires hope among some tweeters.

Finally the concept network for “happy” showing a quite different picture:

The top tweets containing “happy” are not about the elections – these elections are nothing to be happy about according to Twitter - but are by young girls, talking about their moms, and Justin Bieber. It is interesting that the term “happy” does not even show centrally in the term network.Rather, the discussion is about things that make these girls, who are mostly not even from the US but from Asia, happy, like love, and surprisingly, their moms, and it seems, eating a Burger at MacDonald’s in San Francisco.
The only election tweets are by girls rejoicing that voting is finally over, so they don’t get the weird calls at their doors by election workers anymore.

Tuesday, August 31, 2010

Emotions Draw Close Friends: Analyzing the Social Network Structure of Facebook Fan Pages

Recently we were wondering if the social network structure of fans of a brand, a star, or a cause tells us how passionate the fans are. To be more precise, we were looking at the network structure of the friendship network of Facebook fan pages. This means that we collected – as far a publicly accessible – the friendship network of the people who clicked on the “like” button on a fan page.
For a start, look at the fan page of our own COINs2010 conference (by the way, the conference will be soon in Savannah Oct 7 to 9, at SCAD, we hope to see many of you there ☺ ).

The dark dots in the network are the fans of COINs2010, the green dots are their friends. This means that for this initial analysis we looked at how many and how well-connected friends a fan of COINs2010 has. We ignored direct links between the fans, but focused on their external friendship network.

In this first attempt we looked at a total of 15 fan groups in 5 categories, see the table below:

We (admittedly subjectively) ranked the emotionality from 1 (product brands) to 5 (medical causes). We found positive correlation of 0.33 (although non-significant) between the network density and emotionality. This means, the more connected the friends of a cause or brand are, the more emotional they are about their cause. Even more interestingly, we found significant negative correlation between the clustering coefficient of -0.57. This means that the more the friends of fans are clustered in subgroups, the less emotional they are.

The conclusions would be that the causes with the most emotional supporters have a dense, but evenly spread out network, with few clearly separated subgroups.

Based on this admittedly very preliminary analysis, what are actions you can take to further you cause? The answer is simple: Help to weave the network of your supporters.
1. broker connections between supporters
2. fight fragmentation of supporters by connecting subgroups
In short – help build one large happy familiy!

Thursday, August 05, 2010

Predicting Stock Market Indicators Through Twitter “I hope it is not as bad as I fear”

We have been working on trying to predict market indicators for quite some time by analyzing Web Buzz, predicting who will win an Oscar, or how well movies do at the box office. Among other things we have correlated posts about a stock on Yahoo Finance and Motley’s Fool with the actual stock price, predicting the closing price of the stock on the next day based on what people say today on Yahoo Finance, on the Web and Blogs about a stock title.

The rising popularity of twitter gives us a new great way of capturing the collective mind up to the last minute. In our current project we analyze the positive and negative mood of the masses on twitter, comparing it with broad stock market indices such as Dow Jones, S&P 500, and NASDAQ. We collected the twitter feeds from one whitelisted IP for six months from March 30, 2009 to Sept 4, 2009, ranging from 5680 to 42820 tweets per day. According to twitter this corresponds to a randomized subsample of about one hundredth of the full volume of all tweets, as the total volume in 2009 was about 2,5 million tweets per day. We tried to measure collective hope and fear on each day by applying the simple metric of counting all tweets containing the words “hope” – there were 54 to 467 tweets per day, and “fear” or “worry” – there were 9 to 100 tweets per day. This tells us that people prefer optimistic words (hope) to pessimist words (fear or worry).


As external benchmark of investor fear we used the Chicago Board Options Exchange Volatility Index VIX, which is strongly negatively correlated with Dow, S&P 500, and NASDAQ, which is not surprising, as the spread of stock options on a given day is used to calculate VIX. Initially we expected the number of tweets with hope to negatively correlate with VIX, and the number of tweets with fear or worry to correlate positively with VIX. Surprisingly, we found positive weak but insignificant correlation for both “hope” (0.135) and “fear” or “worry” (0.172) with VIX, and negative significant correlation with both “fear” and “worry” and “hope” with Dow NASDAQ and S&P500 (This means that people start using more emotional words such as hope, fear, worry in times of economic uncertainty. We therefore created a simple twitter-volatility index combining mentions of hope, fear and worry, normalizing it with the total amount of tweets per day as a baseline. This index displays strong significant negative correlations to Dow, NASDAQ and S&P500, and strong significant positive correlation to VIX (see table below).


The picture below visualized the negative correlation between Dow (blue) and “hope, fear, and worry” (green) in the period March 30, 2009 to Sept 4, 2009.


To put this in simple words, when the emotions on twitter fly high, that is when people express a lot of hope, fear, and worry, the Dow goes down the next day. When people have less hope, fear, and worry, the Dow goes up. It therefore seems that just checking on twitter for emotional outbursts of any kind gives a predictor of how the stock market will be doing the next day.

Just to be clear, what we have presented here are very early preliminary results, and much more work is needed to scientifically verify it.

Monday, July 26, 2010

My new Coolfarming Book out

I am delighted to announce that – finally - my new book
"Coolfarming - Turn Your Idea Into The Next Big Thing" just came out.

“Coolfarming” is about how to grow your own trends by creating an environment where COINs (Collaborative Innovation Networks) flourish; then - once a product has become established - extend the creative pool into a Collaborative Learning Network, or CLN, whereby a targeted group of interested people are brought in to learn the basics of the product, make suggestions for improvements, point out deficiencies, and push the idea forward.
When this feedback gets incorporated, things get really interesting, expanding the process further outward to a Collaborative Interest Network (CIN) that encompasses thousands or even millions of users, building what hopefully turns into a loyal fan base…and virtually guaranteeing the success of the idea.

Based on case studies and examples from Linux to the Twilight series, from Procter & Gamble to Apple, this book lets you in on the practical, step-by-step processes that will allow you to successfully cultivate the kind of swarm creativity that generates hot new trends . . . and then push them over the tipping point to commercial success.

Get it from the publisher
Get it from Amazon

If you are interested in hearing about it firsthand, I will be teaching a workshop about coolfarming at the 2nd International COINs (Collaborative Innovation Networks) 2010 conference in Savannah, it would be cool to see many of you there.

Insightful review by Barry Richardson

Thursday, June 24, 2010

Predicting the World Cup 2010 Winner

Today I was giving a presentation about Coolhunting through Swarm Creativity at the CRM Forum in Zurich. As there is currently Soccer World Championship time in South Africa, at the end, the moderator Susanne Wille asked me to make my predictions about which team would win the World Cup. Unfortunately, reading the collective mind on the Web does not predict the outcome of 22 soccer players fighting each other particularly well, not to speak about inexplicable decisions of the referee. I therefore refused to make a prediction. Nevertheless, of course we have our prediction system running on our new CoolTrend 2.0 for the last few weeks, because the wisdom of the crowd is still better than chance, although not on the level of accuracy we can reach when predicting political elections or movie box office returns.
So here are our trend curves about which team will win the World Cup, as of June 24, 2010, first the trends on the Web, then on the Blogs


The first thing to note are the huge oscillations among the leaders. Even Italy, out by now, has been traded once (around May 18) as a leader. Currently (June 23rd), the crowd both on the Web and Blogs thinks that Argentina and Brazil have the best chances to be the 2010 Soccer World Champion. Well, there are still many - unpredictable - things that can happen until we will know at the final, July 11, in Johannesburg.

Sunday, May 09, 2010

When are we ready for eternal life?

Well, I guess most of us would like to live forever. At least that’s what I think when I see how old people are clinging to their lives. On the other hand, if all of us would live forever, while producing more offspring, earth would soon overflow. So the solution, if we get eternal life, would be to have no children anymore.

This, however, is in contradiction to Darwin’s evolution. We need to reproduce, to mix our gene pool, and adapt to the changes in environment. Evolution is brutal, too. It’s all about survival of the fittest, of trying to beat the competitor and make sure that my own genes reproduce. Mankind is no exception among the other species. The history of mankind is a history of wars, of killing one's enemies. Today this has been ritualized; the Geneva Convention describes what’s allowed and what’s not. But this is still far from perfect, frequently broken, abused, or ignored.

The conclusion is, then, that we will be ready for eternal life when we will have reached perfection - no need for evolution anymore. In perfect state there is no need for competition, for beating or killing the competitor anymore. Unfortunately, we still have a long way to go. But if we can get rid of aggression, of getting our satisfaction from working together to create new things instead of competing against each other, we have come at least a little bit closer to perfection.

Thursday, May 06, 2010

50 best blogs on creative thinking

This morning I got a nice e-mail from Anna Miller, alerting me to the fact that the swarmcreativity blog has been listed in her post on "50 Best Blogs on Creative Thinking". I quickly looked at them, excellent selection. Thanks Anna!

Sunday, April 18, 2010

The world’s greatest coolfarmers – what we can learn from Jazz

Yesterday evening I was part of a great coolfarming experience. It was the final performance of Jazzaar, a one week workshop of practical music for talented young musicians culminating in public concerts on Friday and Saturday evening. Jazzaar is the brainchild of Fritz Renold, a Jazz musician, composer and music teacher. Every year since 1998 Jazzaar invites internationally renowned Jazz musicians to Aarau, where they teach and play with the most talented and motivated young musicians age 14 to 24 of the Canton of Aargau.

For me how Jazz musicians play together is a great blueprint for how creative teams should work together. In Jazz, improvisation isn't a matter of just making old things up. Jazz, like any language, has its own grammar and vocabulary. There's no right or wrong, just some choices that are better than others.
According to “A Passion for Jazz”…”Jazz players will choose phrases that seem to be preordained so they intuitively know where they are going, even though it's being created at the instant they are hearing it. The musicians are actually spontaneously creating a very intricate form of theme and variation; they all know the tune and the role of their instrument. The guitar, piano, bass and drums, while all able to solo, basically provide the rhythm and harmony over which the soloist will create improvised variations. The structure is flexible so that the soloist may venture in various directions depending on the inspiration of the moment.”

With their rotating lead among the soloists, this model of collaboration - based on a common language - is precisely how COINs should operate.

Yesterday Saturday evening was the final concert for this year, in the sold out KUK (Kultur&Kongresshauss) Aarau. As the official guest of honor, the Swiss President, Doris Leuthard, was there with members of the government of the Canton of Aargau and the City of Aarau.

It was a great evening, with the band made up by famous professionals such as Buster Williams, Corey Allen, Mark Gross, and half a dozen others. The conductor, Kevin Fields, was marvelous. But my vote for greatest coolfarmer of all goes to the singer, Roseanna Vitro, who did a tremendous job integrating the audience and sharing credits for all the pieces she performed with the other soloists. The soloists were always made up both by famous stars and young musicians from Aargau. The climax came when, after frenetic applause, she led into the encore. She asked the audience to sing along, which we did soulfully. And then she asked the young soloists to improvise along with the old experts. It was a decisive moment for the young artists, because all the other pieces and solos they had played until now they had carefully practiced the previous week. But at the end of the concert, with an enthusiastic audience, the young players dared unlocking their own creativity, and one young soloist after the other, bass, sax, piano, trumpet joined in, taking control of the performance for a short, but infinitely intense moment and then passing on leadership to the next.
Coolfarming at its best!

Saturday, March 20, 2010

What Motivates Creators – Lessons from a Cool Artist

Little did I know when I was attending one of the evening events of Swissnex that I was about to meet a truly impressive creator. After filling a heaped plate of food at the sumptuous buffet after the talks, I was looking for a place to eat when I noticed two attractive women in the back of the room. As soon as we started talking I noticed that they were not just a pleasure to look at and talk with but real creators. In my discussion with Magdalena, she told me that she was a graphic artist producing digital art. When she invited me to visit her in her studio I jumped at the opportunity.

On a sunny afternoon the next week I climbed the stairs to her studio in Revere outside of Boston. The studio doubles as a gallery, displaying really cool art. When we sat on her couch in the middle of art books and framed digital pictures, Magdalena told me a bit about her life. She was born in Poland to parents of German descent, as a young adult she moved to Germany, later after she had married an American she moved to Boston, where initially she continued working as a biologist for a research company. Ten years ago, however, she decided to quit natural science and to exclusively focus on art. In her art she takes digital photographs of people, dressing them in medieval cloth and embedding them in a renaissance style environment, giving her digital pictures the feel of Rubens or Boticelli paintings. For her digital paintings she needs lots of models, whose pictures she takes on the floor of her living room. She has a wide circle of models and friends, drawn from fellow artists, painters, photographers, and actors but also including scientists from her previous work. One of the more interesting ones is Niki the crossdresser with large wig at night - Nick the scientist during the day.

Her masterpiece until now is “the sleepers” (extract pictured below), a digital composition of 35 models, combining 35 digital photographs of 25 different people into a surreal nightly landscape of ethereal beauty.

When I asked her how she finances her art, she told me that she works seven days a week, only now it’s not work anymore but her passion and labor of love. She explained “I want to spend my time to become a better artist, and not wasting my time selling art”. This of course means, that besides creating art, and occasionally selling art, Magdalena is doing all sorts of odd jobs. She has folded towels in “bed & bath”, and she worked as a temp usher during the entire last stay of Cirque du Soleil in Boston. During these jobs she never stops promoting her art. While working at Cirque du Soleil, she produced one of her signature large digital artistic creations with some models from Cirque du Soleil, and asked for permission to put it up in the tent. When her temp manager denied the request she found another employee of Cirque du Soleil who put it up for her in the break room of the internal employees of the Cirque.

She even succeeded in having some of her art shown in the ICA – the Institute of Contemporary Art in Boston. This happened when one day she learned that next Saturday morning the ICA would show 100 pieces of art from local artists. Needless to say she was the first one at the door next Saturday at six in the morning, not only assuring a premium spot for her piece of art, but even winning an award for the best picture.

For a change, Magdalena likes to go out to art shows and cultural events like the one from Swissnex at the Harvard Business School where we met. Another favorite of hers are the events from the Goethe Institute in downtown Boston. Not only is there good food at these events, but also good company, and the opportunity to recruit more models for her masterpieces, and perhaps even a customer or two.

When I asked her what motivates her, the first thing I noticed was a fierce sense of independence and self-determination. This reminded me of the famous quote of Perikles, politician during Athen’s golden age of democracy: “Make up your mind that happiness depends on being free, and freedom depends on being courageous”. Magdalena put her own principle into the following words “You end up where you want to go if you let go without knowing how you will get there.”
Her main motivation to do what she does is twofold: seeing the happiness of people when they acquire her art, and the big satisfaction she draws from the process of creating new art.

One of her ways of increasing dissemination of her art is to choose her models well. If they like her art and their own representation in Magdalena’s pictures, they will spread the word. Usually this works very well, at least it worked for me. At the end of my visit she had successfully recruited me as a model for her next masterpiece, tentatively named “the wall”, which will combine a large number of people building up and tearing down a wall.

I think every member of a COIN can learn a lot from Magdalena!

Saturday, March 06, 2010

Oscar Predictions Updated

Here is an updated analysis of the Oscar predictions, run on March 2nd, 2010. Again, sources being used were the collective mind of movie enthusiasts on IMDb.com and movie fans from all around the world represented by bloggers and Web masters. The Internet Movie Database (IMDb) was analyzed using a specialized version of sentiment retrieval with an adapted bag-of-words. Web and blog buzz values come from Cooltrend. Tapping the thoughts of this swarm delivered the following update to our older analysis:

While IMDb users have two clear favorites, Hurt Locker and Avatar, the Web seems to favor Hurt Locker. Bloggers are seeing 500 Days of Summer in front right now, but this might change since the blogosphere tends to be relatively volatile in opinion trends. So, in sum we would expect Avatar to have the best chances on winning best picture, followed by Hurt Locker.

Directors – here it is again a head-to-head race between Kathryn Bigelow (Hurt Locker) and James Cameron (Avatar). They seem to be tightly correlated with the best picture results and this is probably not too surprising. Interestingly the Web sees James Cameron while Kathryn Bigelow is really not that popular. Blogosphere is undecided in this category, Mr. Cameron is clearly behind in the buzz though.

The surprise nomination of Sandra Bullock for the best actress award caught us last time we made the prediction – including her this time shows that she still is behind IMDb’s favorite Meryl Streep. Looking at actors the following picture rises:

Again IMDb decided on a clear favorite and it is Jeff Bridges. While Jeremy Renner and Colin Firth certainly receive some attention in discussion they are both only half as popular in the forums as Jeff Bridges is.

In summary we now give our final prediction, with a primary (the most likely winner) and a secondary choice, based on the collective mind of the Internet’s movie swarm:

It is really a close decision in best picture and director, but considering the combined Web, Blog, and IMDb metrics there is a slight preference for the movie Avatar to be this year’s best picture winner – Hurt Locker is very close though. Director category discussion sees Kathryn Bigelow receiving the award while James Cameron is put second here. In actors it looks like Jeff Bridges pretty much made the race, however in actresses it is not completely clear yet who will cross the finish line first – though Meryl Streep has a slight advantage. Let us look back at this once the Awards have been given out!

Saturday, February 06, 2010

It’s In a Name - How much is an athlete worth?

These days Forbes again published sport’s top earning athletes. In spite of his transgressions, Tiger Woods still leads the pack with $64 million, followed by soccer player David Beckham ($18 million), tennis star Roger Federer ($16 million), Nascar driver Dale Earnhardt ($14 million) and basket ball players LeBron James ($13 million) and Kobe Bryant ($12 million).
I was curious to see if Web buzz and valuation of a player's attractiveness for corporate marketing executives from Nike, Addidas, Reebook and the like had any correlation. The two pictures below show the Web buzz share of five of the six athletes (I skipped the Nascar driver):

And the blog buzz share:

As the pictures show, the ranking at the top corresponds nicely. Tiger Woods is lonely at the top both on the Web and on blogs, but then there are some interesting differences. Kobe Bryant has more than his share on Web buzz compared to what corporate marketers paid for him, which means they got a good deal considering his high Web popularity. David Beckham, on the other hand, seems somewhat overpriced considering that he commands only 16% Blog and 20% Web buzz, well behind Roger Federer’s 21% and 25%. The conclusion: invest into the Swiss, he seems on the rise right now, at least as far as Web and Blog buzz goes.

Thursday, February 04, 2010

Oscar Predictions 2010 – Official Nominations of 82nd Academy Awards

Oscar Night is getting closer. On Tuesday Hollywood announced this year’s official nominees of the 82nd Academy Awards. As previously posted on this blog we are running our own Oscar predictions based on buzz in the Oscar forum on imdb.com and Web and Blog buzz analysis. Our latest results presented here were calculated on January 5th 2010 which is over one month before the nominations were released. In our analysis we focus on four categories, best picture, best director, best actress, and best actor.
The following table for best picture shows the results of our Oscar-Coolhunting four weeks ago. We computed an Oscar Index for every movie. This index consists of combined Web-, blog-, and message board – buzz and can be interpreted as the relative probability of a movie to receive an Academy Award on March 7th. Our approach simply measures what the Web (this means all of us) thinks. It works by multiplying the “how many” with the “who”, i.e. multiplying what somebody says (in a forum or blog) with her/his influence.


Our top 4 ranked movies correspond very well with the most promising nominations. Hurt Locker, Precious, Avatar, and Inglourious Basterds are head-to-head and clearly leading in our Coolhunting study. The following table nicely shows that these four blockbusters including the film with George Clooney – Up in the Air – also received the most nominations this year.


This year’s nominees for best directing match perfectly with our five candidates predicted through analyzing Web buzz. Kathryn Bigelow and Quentin Tarantino have a decent lead in our ranking. James Cameron with Avatar, however, will definitely be in the game when it comes to the showdown on Oscar Night.


Our predictions in the category for best actress illustrate a clear lead for Meryl Streep (30%) for her performance in Julie & Julia. Hollywood newcomer Gabby Sidibe playing a great role in the drama Precious was also on our list four weeks ago. We are missing Sandra Bullock and Helen Mirren.


The race for best actor in a leading role 2010 is much more competitive and no clear favorite could be identified in the run-up to the awards ceremony. George Clooney is definitely in a favorite position however Jeff Bridges seems to be in good shape this year. Unfortunately, we could not anticipate this at an early stage.


We are very excited to see who actually will win the Oscars on March 7th 2010. We will definitely stay tuned and provide updates as soon as we have new results.
Apropos, since the hype about the Academy Awards all around the globe is always very high we were recorded in our Coolhunting by Swiss TV in January. They will broadcast our story in the show “Einstein” prior to Oscar Night.

Friday, January 15, 2010

Who will become the next Massachusetts Senator

The race to fill Ted Kennedy's vacant senatorial seat in Washington has turned into a real shocker. At the outset, it seemed clear that in deeply democratic Massachusetts only a Democrat could become the next senator. As of today, it seems to be the other way. Republican candidate Scott Brown is ahead of Democrat Martha Coakley by 5 points.

I wanted to know what the Web would tell us. The picture below shows the daily betweenness values of the two candidates, we can cleary see how Scott Brown is gaining on Martha Coakley.

It seems however, that there is a silver lining for Ms. Coakley, at least in the blogosphere.
The next picture shows her centrality as of today (January 15). She is ahead by 6 percentage points.

The last picture, just for fun, shows the Blogpost-Network leading to the percentages. The red dots are the Web sites promoting both candidates. The brown dots promote Mr Brown, but it seems that the green (should really be blue) Web sites carry some more weight, leading to Martha Coakley's slight lead.

Let's wait until January 19, then we will know for sure.

Friday, January 01, 2010

Scientists, Monks and Bankers - It's All About Love

Why do people collaboratively engage in innovative tasks? What’s the motivation to work together to develop something new? After all, if I do everything by myself, I will reap all the rewards for myself, and don’t have to share. Nevertheless, humans are the most social species of all, and progress is only possible by collectively creating new things “standing on the shoulders of giants”, by learning from what others have done, and apply it in novel ways.

According to my colleagues at the MIT Center for Collective Intelligence Tom Malone, Rob Laubacher, and Chris Dellarocas there are three main reasons why people engage in collective intelligence systems:
Money – financial gain is a strong motivator for people participating in markets and traditional organizations, either for direct payment, or for future payment, e.g. acquiring new skills.
Glory – getting recognition and building up a reputation can be an important motivator.
Love – love can be an important motivator, be it because people intrinsically enjoy an activity, because they like to socialize with others, or because they feel they are contributing to a cause larger than themselves.

Randall Collins links together these motivators in his theory of interaction ritual chains. He extends Emile Durkheim’s concept of effervescence, the energy that groups of people experience when they are together at a sporting event, a rave, or a riot. For Collins the mechanisms that drive society are nothing but interaction rituals. Interaction rituals consist of 4 components:
1. the people need to be bodily present at the same place. Through mutual feedback they will charge up the situation with excitement and significance.
2. there needs to be a boundary that demarcates the insiders from the outsiders, giving insiders a feeling to be privileged.
3. all insiders have a shared focus of attention, by communicating this focus to each other they become mutually aware of this focus.
4. by experiencing 1 to 3, all participants share a “common mood or emotional experience”.

When these 4 components combine successfully, participants are infused with emotional energy, the feeling of excitement, achievement, and enthusiasm – in short they are happy! Applying the concept of interaction ritual chains to the above three motivators "money", "glory", and "love" means that "love" as experienced in ritual interaction chains is the key motivator at the core, glory and money are simply enablers to obtain "love". Just look at Tiger Woods, who used his money and glory to buy love.

What motivates scientists?
To explain how love is the main motivator for people interested in collaborative innovation, let’s look at what motivates scientists, people whose profession it is to innovate. At the outset, scientists are motivated mostly by glory: it’s all about publish or perish, i.e. getting one’s papers into highly-ranked journals, and then be quoted by others.
Swiss researchers Margrit Osterloh and Bruno S. Frey - himself a highly-cited researcher - analyze the deficiencies of the current academic ranking system based on peer review and citation count. Currently academics are ranked first by how many papers they get into top-ranked scientific journals. The decision on which paper makes it into a journal is based on peer reviews. The second criterion of success is citation count of their paper, i.e. how many other researchers cite the papers in their own academic work. Unfortunately both of these ranking systems have serious deficiencies, even leading to what Frey calls “academic prostitution”: As has been shown repeatedly, peer reviews have very low to no correlation with future citation, and citation count by other authors is seriously biased towards authors choosing articles they quote to increase acceptance of their own papers. Frey therefore proposes to revise peer review and quote-based ranking to reward intrinsic motivation. After all intrinsic motivation is what usually gets young researchers started on their career path: factors such as curiosity, flow experience in a fascinating activity, self-image, compliance with civic virtues, social or professional norms. In other words, all factors based on love with what they do.
While scientists therefore make a good role model for people involved in collaborative innovation, there is an even better one, which has been tried and tested for well over a thousand years, and has been analyzed by the same Bruno S. Frey: Benedictine monasteries (recently profiled in NZZ).

Why monks are coolfarmers (c)
In a paper on management principles of monastic life, Katja Rost, Emil Inauen, Margit Osterloh and Bruno S. Frey study corporate governance at 134 Benedictine Abbeys in Southern Germany and Switzerland with an average age of nearly 500 years per monastery. It turns out COIN members can learn a lot from the monks. Benedictine monks organize their monastic life around three pillars:
Self-management: normally monks choose their own abbots from among their own ranks. Among the democratically elected abbots, only 11% turned out to be incapable, while among the externally imposed abbots, 67% were judged bad leaders. The Benedictine monks also have a lot of say in daily monastic life: there is a democratically elected “executive board” working with the abbot, a “consilium” advising and supervising the executive board, and there is also the full “parliament” of all monks convening multiple times per year and making fundamental decisions.
Core value system: Benedictine monks adhere to core values such as fairness, justice, mutual respect, mutual agreement, and forgiveness. Monasteries invest a lot of time into carefully selecting their new brothers, which join brotherhood over a well-calibrated multi-step multi-year process. Spending a lot of time with older monks, the young learn core values by example from the old. Additional learning happens through institutionalized readings during the meals.
External control: All the Benedictine monasteries are members of the congregation, lead by the archabbot. A visitation committee from the congregation inspects each monastery every five years, checking financial and spiritual health, with focus on advice, not on control.
Daily life in the Benedictine monasteries is governed by core values instead of control. This autonomy and self-organization increase intrinsic motivation, leading to higher quality results. Comparing average lifetime of a monastery (500 years) with the lifetime of large companies, which rarely survive 50 years, monks easily beat company executives. Love, the main motivating factor in Benedictine abbeys, therefore again trumps money and glory as corporate motivator.

What motivates creativity?
So far it seems that “love” is the main motivating factor for collaborative innovation. So is there a difference between love-driven “good creativity” and “bad creativity”? Creativity defined as “creating new things by combining existing things and processes in novel ways that have not been done before" is neither good nor bad.
But thinking this through further raises the question if e.g. creating a new derivative product for a Wallstreet bank is creativity also. After all, the Wallstreet banker is applying existing thinking to a new area to create value – mostly for himself. And that’s why I think that this is NOT sustainable creativity. The Wallstreet banker is entirely extrinsically motivated – he does what he does to make a lot of money, preferably for himself. Enron employees were highly creative in trading energy options, and even more creative in gaming the Californian energy system – all to create value, towards the outside for Enron and its shareholders, in the end, however, it all boiled down to optimizing their own bonuses. As history has shown, this was not and will never be sustainable! For some time, Enron revenue and share price went up, all to come down with an ugly crash after a few years. All that Enron employees did, was create one heck of a Ponzi scheme. As it now turned out, the real estate bubble in the US, created by “genius traders” at Wallstreet, was just yet another Enron, only at a vastly larger scale.
Enron employees did not break the laws of the United States, what they broke is the laws of good and ethical behavior. The schemes the Enron employees cooked up might have been legal, but they ripped off the majority – all of the users of electricity, i.e. all of us, to the advantage of a tiny minority: firstly the shareholders of Enron, and more importantly, the inventors of the energy trading systems.
And that’s why free markets don’t work. The underlying premise of Adam Smith’s invisible hand is that if everybody is off to fend for himself, trading on an unrestricted market will make society at large to be better off also. Well, this would only be true if there were total transparency, and no law of the bell curve, i.e. of smarter people trying to profit from not-so-smart people. And that’s why I think that only collaborative innovation motivated by “love” leads to sustainable progress.

What is better: banker or monk?