Category Archives: Uncategorized

Statler at CSCW 2010

Statler & Waldorf Do you know these guys? They are described asThey are two ornery, disagreeable old men who…despite constantly complaining about the show and how terrible some acts were, they would always be back the following week in the best seats in the house.” Looking at their snark in aggregate, one finds them to be particularly noisy when Fozzie Bear performed. Early last summer, I began to wonder if now-a-days they would be tweeting snark during a show.

Fortunately, people have stepped up to fill the void and tweet while they watch tv. So last year I began investigating people tweeting during live events/performances in order to discover interesting moments, people’s sentiment, what people are talking about, and media segmentation. The Statler prototype embodies most of my findings to date:

Statler Screenshot

The prototype has two modes: Debate 2008 & Inauguration 2009. Based on a sample of tweets from the first debate of 2008, Statler automatically identified 9 topic segments which align to CSPAN’s editorial slices with an accuracy of 93%. You can also see the trending tweets in comparison to top terms from the debate speakers (taken from the closed captioning). For the Inauguration, Statler uses 50,000+ tweets taken from the public timeline to give a more ‘real-time’ feel to how the crowd is moving as the tweets, the tweet structures and terms change over the course of the swearing in and the speech. Of note here is Statler identified the moment of swearing in as the most interesting point during the 30 minute Inauguration video as well as identified the messing up of the oath as something which was conversationally interesting. The latter will not result as a salient term using a conventional vector-space approach.

Feel free to try out the demo and say be sure to say hi if you’re at CSCW. Look for me in the Horizons and Demo programs. If you can’t find me, look for Naaman who has a good line of sight to spot people in the crowd.

Milgram to TagMaps like Lynch to Flickr Alpha Shapes

After we came up with Tag Maps at Yahoo! Research Berkeley, Morgan Ames (then one of our star interns) pointed out the surprising similarities to a study that was done 30 years earlier, by Stanley Milgram, the famous social psychologist. In his study, Milgram asked 30+ participants to list names of attraction in Paris. He then visualized these on a map, in a size according to the number of times each was mentioned. Here are the automatically-created, Flickr-based TagMap of Paris (based on geotagged photos taken in that area), and the same exact area as represented by Milgram’s visualization.

tagmaps paris

milgram paris

I have been showing both these images in my talk for a while now — can’t seem to get sick of them, even if my audience might just be…

I’ve also been talking for a while on how we can use the aggregate contributions on Flickr to mark boundaries of geographical objects, such as, say, neighborhoods, using all the photos tagged with a neighborhood name. Talk is cheap, but the smart people at Flickr not only figured out how to do it (with slightly different data than tags) but also released the data and the source code for anyone to use. Blame Aaron Cope and Rev Dan Catt.

Well, here’s the thing: turns our a famous scholar also beat Flickr to it, some 40 years ago. Kevin Lynch, in his groundbreaking essay/book The Image of the City, collected people’s descriptions and hand-drawn maps of three cities (Boston shown here, also Jersey City and downtown LA). In one study, he extracted the “maximum boundaries” for each neighborhood as drawn by all the interviewees, and plotted them on the map.

Here are the automatically-created, Flickr-based map of Boston Neighborhoods, visualized using the excellent Tom Taylor’s Neighborhood Boundaries, and Lynch’s maximum boundaries of neighborhoods in the same area.

Neighborhood Boundaries for Boston

Lynch's neighborhood boundaries

I have pre-ordered Milgram’s book of essays, to arrive in February. Might as well find out what’s there before we re-invent something else!

Yes, Virginia….

@santa: why are the chichfilas all closed?

Ever wonder who you are talking to? Or who’s talking back? Recently I came across Mentionmap by asterisq. It shows you a nice little viz of who any Twitter User has mentioned. Check out aplusk or naaman. The viz itself is quite nice, it shows you people, hashtags, and the stroke denote link degree.

Six months ago, I took a look at a mention map of some tweets I captured from the first presidential debate of 2008. Instead of examining in/out degree, I chose to take a look at eigenvector centrality (EV). If you dont know what that is, think PageRank in a social graph (actually PageRank is variant of EV). In such a network, EV shows you the most salient node. For example, the yellow brick road lead to the Emerald City. Presumably, other towns had more roads in and out of them (higher in/out degrees) but they all lead to the Wizard’s city (which had an degree of 1). EV Centrality would rank the Emerald City as the most salient city in all of Oz. Lets take a look at the debate tweets:

In this graph, the node’s size is relative to its EV Centrality. The larger the more salient. Clearly there is a cluster of importance. Lets take a closer look:

Obama, NewsHour and McCain take the top three spots.

The debaters and the moderator had the highest centrality. Despite not having the highest degree. Barack was significantly more salient than Jim or John. When I examined just the degrees of this network, the main characters became less important and we picked up on chatty micro-bloggers.

Think Santa knows about this? Is his list rank ordered? Does he cluster who’s been naughty? Should he be using NodeXL?

Reality Update: People Still Watch Live TV

Not just live TV, but people still tune in to catch their regularly scheduled programming. I’m not sure how Naaman does it in NYC, but I like watching Top Chef when it airs. I’ve been looking at social interactive TV for some time—thinking and building new interfaces for live TV watching and social sharing. Just about anyone I talk to about this work, at least in San Francisco, says “Oh I never watch TV”. If they do watch TV they say “this is useless, everyone uses a DVR now”. I do <3 my TiVo, but really I’d rather tune in on time to watch my stories.

Hulu and DVR addicts should know about this recent Nielsen study. The study found on average 1.15% of all TV watched is time shifted via a DVR. Tiny tiny tiny. What’s even more crazy is that 1.15% is up 21.1% from last year. This means for any month, 7 hours of your TV is time shifted.

You can read the full report; but I’d rather you tell me: Do you time shift your TV? If so how much?

Huffington Post: I know you watch TV News and I know you like it

I don’t actually read the Huffington Post all too often. Usually, I follow it and other reporting agencies like it through social network streams like Facebook or Twitter. If you know me already, you know I have issues the use of the term social there, but it’s time we paid it some due attention. Social streams let us know what our friends want to share (and converse with us…yes Naaman, I still say social media is about conversation). Additionally, we can also follow trusted sources, celebrities, or agencies we want specific news from. I personally enjoy this filtration, which accounts for probably 40% of my news consumption; the rest comes from TV and general news (paper or web) reading/viewing. TV and videos themselves are highly social activities—weather we watch them together or just ask our friends if they saw it later. This is what lead me to create Zync for Yahoo! Messenger and what leads others to create similar technologies.

This weekend, CNET Senior Editor Natali Del Conte, who I follow on Twitter, posted a link to the following “fair/unfair” story about broadcast news from the Huffington Post. This caught my attention, so I followed the link to the article.

The article starts in a rather pointed tone:

American television news is returning to its roots as an information wasteland. Pretty faces with largely empty heads read teleprompters and mug for the camera. A dollop of information surrounded by a thick sugar coating of Kewpie doll. The major difference between the evening news and Jeopardy is that Alex Trebek is probably better informed.

Which leads one to think this is an op-ed…remember when those were on the last page of the paper? I’ll let this point slide for now. The author continues:

Television is still the dominant source of news for most Americans.

Immediately you can tell the author (Brian Ross) is upset. From some recent studies he points out, half of us get our news from the TV first. If we find something of interest, 29% of us will hit the Internet to learn more. But actually, 48% of us will watch more TV for followup reports. My guess is if you want to follow up a month later, you’d likely hit the web. In the chance you want late-breaking news online, you’ll hit Yahoo! News way before you check the HuffPo.

All this reads well and fine to some extent, but actually, he is upset that many great reporters cannot survive on TV. At the same time he’s cites TV News as a degrading trend of pandering to the ignorant TV audience “rather than trying to lure back the hard-core news junkies”. There’s an interesting slight of hand in this argument (I think, more formally, cum hoc ergo propter hoc).

He describes TV news agencies as “in a live feed where news is breaking, they buck-and-wing while research staffs scramble to Google up information to make them look a little less piteous….as Jon Stewart so aptly point out in his recent rip of CNN, that they don’t even bother doing any fact-checking”

Which he then illustrates using a Daily Show clip. Yes, a Daily Show clip. In his rather long argument about print (or web rather) being collected, thought-out, and real, he embeds an 11 minute and 33 second video from a comedy TV program to support his argument. Way to go. I’ll let you read the whole rant which is worth the look. It seems his account just falls apart despite a nice collection of sources (Natali’s point is correct, it is a mash of the fair and the unfair…I’ll just point out that the threads are orthogonal at best).

This article did lead me to think about is TV and its social nature. I wonder of the half of us who watch it for news…why? do we actually want fodder for hard-core news junkies? or do we want the mix and balance we get? more so, are we watching this news with other people? do we ask our friends “did you hear what happened in Gaza?” as equally as “did you see Bon Jovi on the Today show?” The Internet or even print won’t magically become a primary source without a real social presence (and I don’t mean add a ‘tweet this’ button to your article either). But maybe there is a more effective way for the HuffPo to increase that 29% followup if only there was a socially viable method.

Maybe people reading this blog don’t watch TV or maybe you know what just happened in the mideast and you saw Bon Jovi on Today. What do you do when you want to follow up from a TV news story?

Google Wave: one ? transition at a time

Like Naaman, I was excited to hear about Google Wave. I signed up for the Sandbox access to hack on it. I signed up for ‘Wave Preview‘ to see a more stable version. Finally, once things ironed out, I decided to start building widgets.

Having worked on synchronous web interactions for sometime, I was happy to find the overall API to be pretty clean. The overall idea is simple. When you make a gadget that does something, have it submit that event as a change in state (they call it a delta). Quite simply, if you click a button and that button increments a shared counter. The OnClick handler for that button should call something like:

wave.getState().submitDelta({'count': value + 1});

Then you implement a function, lets call it onStateChange() which will check for the value of count and set the counter accordingly. Each delta makes a playback step, which in their own API words:

Work(s) harmoniously with the Wave playback mechanism

So, if somebody wants to playback the wave, they start at time 0 and drag the slider to time 20. The onStateChange handler will fire, and the counter will be set to whatever the value was at that point. Something like:

div.innerHTML = "The count is " + wave.getState().get('count');

Pretty neat right? Well not exactly. This works for a simple example. However, if your gadget does something more complex (such as load and unload flash objects), this will cause you some trouble if you aren’t careful. Lets take this example:

  1. I start a wave and add my gadget
  2. The gadget loads some flash
  3. I interact with the flash object
  4. The gadget loads a new piece of flash (overwriting the previous)
  5. I interact with the new flash object

If I play back this wave and jump from step 1 to step 3, I have to perform step 2 and then step 3. Some what similarly, if I jump from step 1 to step 5, I have to perform step 4 and then step 5. This is because if we just jump to step 5, there is no flash object loaded to interact with; the wave will be in an undefined state (and will make the JavaScript from step 5 quite unhappy as it references a null object).

The solution here is to make sure your wave.getState() object has all the information it needs to optimally reconstruct any arbitrary state. So, from our past example I’ll list the state as {key:value, ...} pairs:

  1. {}I start a wave and add my gadget
  2. {load: object1}The gadget loads some flash
  3. {load: object1, action: action1}I interact with the flash object
  4. {load: object2, action: null}The gadget loads a new piece of flash (overwriting the previous)
  5. {load: object2, action: action2}I interact with the new flash object

Each step now clearly contains everything it needs to rebuild the world, without running through all of history again. Also notice step 4 clears out any action that is not applicable to the newly loaded object. This will add some considerable code to your stateUpdated() function (especially since Flash loads asynchronously, you’ll have to wait for a series of callbacks to properly restore the state) but then you’ll get harmonious playback.

If you want to do something fancy like maintain a stack or a more so Turing-complete series of tapes, you’ll have to talk to @hackerjack60 if you can.

Still watching the TV?

Really, does anyone actually care to watch the TV anymore? The latest influx of TV becoming social is bringing a variety of apps and funky visualizations. Take in point the MTV VMA. Why attend & watch when you can tweet? Presumably in the breaks when we were fetching a drink or trying not to spill the sour cream of a crisp. Couple this with iJustine and a soda pop like viz of people and well:



This lets people (including iJustine the hostess) pick people floating up and see terms-live. I would have liked to see this filtered using Eigenvector Centrality; one could find the salient people in the conversation easily.

But, if you are like Naaman, you are probably either talking about yourself or want to hear what people are broadcasting. Got an app for that? Well Frog kinda does, its called tvChatter (coming soon):

You don’t have to configure your favorite tweet app with filters for # tags. Just find the show and follow the tweets, or tweet away! As people chatting about tv media becomes more and more real time, it actually shapes and changes what we know about people using Twitter (remember when it was a social microblogging platform like what two years ago?).

So is this all becoming that all too intrusive computer interface for ‘learning about artwork’ that they give you on a hand held when you probably should just be looking at the painting? Or is everyone happy typing while watching? Naaman? You get a TV yet with actual channels?

The new social face of multimedia tagging.

I’ve never been too concerned with definitions—early in my graduate career I realized they were more often used for turf wars. Just as George Carlin fought to get a definition of what he could or couldn’t say, he showed us a description can be way more powerful. Lately, I’ve been describing quite a few things around people tweeting while watching TV or when at a concert. Currently, there are several great studies characterizing Twitter users. Less concerned with this, I was wondering, “if everyone watching the superbowl tweets what they think about whats happening, what does that say about the sporting event itself” (from a classic Multimedia perspective).

Using a sample of tweets captured from the first presidential debate, I began to investigate if conversationally, people behave the same way as they do when they watch TV. It turns out they do; my colleagues (Lyndon and Elizabeth) and myself were able to topically segment the the first presidential debate and identify the main people in the video, all by looking at a collection of tweets captured from the 90 minutes of the debate.

There are many gritty details (including the usage of Newton’s Method and Eigenvector Centrality) in the full paper to be presented at ACM MM’s first workshop on Social Media. Aside from methodology, we are suggesting there is more to media annotation than explicit tags on Facebook or Youtube. In fact, if Naaman tweets “I miss Paula #idol” while watching American Idol, he is leaving a comment/annotation on the media…despite there being no proper URI where Idol exists (yet!).

Recently, I was invited to speak at Stanford’s MediaX workshop on Metrics. At first, I was curious why I was there, I don’t think of metrics in my day to day life. I think about people and experience and stick figure drawings depicting the negotiation of meaning.

However, if we think about social behaviors and media (and now they relate to uncaptured events in the world): the methodological research becomes an exercise in metrics. What is happening? Is there a singular source event (or a plurality of events)? What do we measure? What does it mean to the source event?…I could go on. But you, gentile readers, can just read the paper or say hi at ACM MM in a few weeks or wait till I post more details about the work.

Social Media Definition (redefinition, that is)

Ayman, I know you’re sick of it by now, but I am revisiting a popular theme for this blog, “What is Social Media”. A definition of social media was attempted (by me) here, and I later added a note about a practical definition of social media in the context of teaching an interdisciplinary class on the topic.

So now, after teaching the first session of that class, let me try again. The following definition will try to broadly scope the topic as described in my Social Media class. But I also believe that this would make a good working definition of this widely-and-wildly-used phrase.

In this definition, I try to follow closely the original meaning of both “social” and “media”. Media is defined as:

the main means of mass communication (esp. television, radio, newspapers, and the Internet) regarded collectively. (Apple Dictionary)

And Social:

needing companionship and therefore best suited to living in communities “we are social beings as well as individuals.

These definitions are echoed in the following, although did not directly dictate it.  Social media, then, is any media that supports these two characteristics:

  • Posting of lasting content in public/semi public settings within an established service or system.
  • Visible and durable identity, published profile, and recognized contribution.

This definition would then include Facebook, Flickr, Delicious, MySpace, Yelp, Vimeo,, Twitter, Dogster, YouTube and their many, many likes.

The definition does not include purposely excludes old media services that allow for comments from users (no durable identity); Wikipedia (no “recognized contribution” that is easily associated with a user); or say, mobile-social applications (no posting of content => not a media!). The definition also does not newsgroups and discussion forums (no published profile, no expectation of lasting content). And it does not include communication services like IM and email that are not public, not even semi-public in nature.

Does this make sense?

On Media Multitaskers

(sidenote: I find that I now blog thoughts that are too long to fit in a tweet; so feel free to follow my tweets, or Ayman’s).

A recent article in the PNAS was quoted in quite a number of media outlets (Hindustan Times gave the Masters student responsible a PhD as well as professorship). From the article, Cognitive control in media multitaskers, by (the formidable team of) Eyal Ophir (get a Web page!), Cliff Nass and Anthony Wagner:

Results showed that heavy media multitaskers are more susceptible to interference from irrelevant environmental stimuli and from irrelevant representations in memory.

Heavy multimedia multitaskers (HMMs) are identified by a survey about media use, and compared to low multimedia multitaskers (one standard deviation over vs. under the mean of the index). The paper compared HMMs (not sure they are aware of the other meaning of the term) and LMMs on a number of tasks, finding that:

individuals who frequently use multiple media approach fundamental information processing activities differently than do those who consume multiple media streams much less frequently: their breadth-biased media consumption behavior is indeed mirrored by breadth-biased cognitive control.

In other words, those who multitask are not effective multitaskers – it’s the opposite. Of course, there are still outstanding questions:

  • Causality: what is the direction of influence? Do HMMs (I still find it hard to use this acronym) tend to breadth-biased consumption of media because of their distraction?
  • Index validation: how robust is the survey and metric created to capture the “media multitasking” index? Do survey participants’ self-reports actually attest to their real behavior, and does the survey really capture “multitasking” or something else? The authors note that Media multitasking as measured was correlated with total hours of media use — maybe that’s what was measured?
  • What other factors are in play? As the participants were all Stanford students, I do not expect major age, economic or education gaps; the also authors tested for differences in a number of dimensions (SAT scores, performance on a creativity task, ratings on extraversion, agreeableness, conscientiousness, neuroticism, and openness and others) and found no significant differences between HMMs (grrrr, acronym!) and LMMs. Does this cover it or is there any other factor that will help explain the differences?

In any case, interesting study — I am looking forward to the follow up work. And now, off to another media!