Google Wave: one ? transition at a time

Like Naaman, I was excited to hear about Google Wave. I signed up for the Sandbox access to hack on it. I signed up for ‘Wave Preview‘ to see a more stable version. Finally, once things ironed out, I decided to start building widgets.

Having worked on synchronous web interactions for sometime, I was happy to find the overall API to be pretty clean. The overall idea is simple. When you make a gadget that does something, have it submit that event as a change in state (they call it a delta). Quite simply, if you click a button and that button increments a shared counter. The OnClick handler for that button should call something like:

wave.getState().submitDelta({'count': value + 1});

Then you implement a function, lets call it onStateChange() which will check for the value of count and set the counter accordingly. Each delta makes a playback step, which in their own API words:

Work(s) harmoniously with the Wave playback mechanism

So, if somebody wants to playback the wave, they start at time 0 and drag the slider to time 20. The onStateChange handler will fire, and the counter will be set to whatever the value was at that point. Something like:

div.innerHTML = "The count is " + wave.getState().get('count');

Pretty neat right? Well not exactly. This works for a simple example. However, if your gadget does something more complex (such as load and unload flash objects), this will cause you some trouble if you aren’t careful. Lets take this example:

  1. I start a wave and add my gadget
  2. The gadget loads some flash
  3. I interact with the flash object
  4. The gadget loads a new piece of flash (overwriting the previous)
  5. I interact with the new flash object

If I play back this wave and jump from step 1 to step 3, I have to perform step 2 and then step 3. Some what similarly, if I jump from step 1 to step 5, I have to perform step 4 and then step 5. This is because if we just jump to step 5, there is no flash object loaded to interact with; the wave will be in an undefined state (and will make the JavaScript from step 5 quite unhappy as it references a null object).

The solution here is to make sure your wave.getState() object has all the information it needs to optimally reconstruct any arbitrary state. So, from our past example I’ll list the state as {key:value, ...} pairs:

  1. {}I start a wave and add my gadget
  2. {load: object1}The gadget loads some flash
  3. {load: object1, action: action1}I interact with the flash object
  4. {load: object2, action: null}The gadget loads a new piece of flash (overwriting the previous)
  5. {load: object2, action: action2}I interact with the new flash object

Each step now clearly contains everything it needs to rebuild the world, without running through all of history again. Also notice step 4 clears out any action that is not applicable to the newly loaded object. This will add some considerable code to your stateUpdated() function (especially since Flash loads asynchronously, you’ll have to wait for a series of callbacks to properly restore the state) but then you’ll get harmonious playback.

If you want to do something fancy like maintain a stack or a more so Turing-complete series of tapes, you’ll have to talk to @hackerjack60 if you can.

Still watching the TV?

Really, does anyone actually care to watch the TV anymore? The latest influx of TV becoming social is bringing a variety of apps and funky visualizations. Take in point the MTV VMA. Why attend & watch when you can tweet? Presumably in the breaks when we were fetching a drink or trying not to spill the sour cream of a crisp. Couple this with iJustine and a soda pop like viz of people and well:



This lets people (including iJustine the hostess) pick people floating up and see terms-live. I would have liked to see this filtered using Eigenvector Centrality; one could find the salient people in the conversation easily.

But, if you are like Naaman, you are probably either talking about yourself or want to hear what people are broadcasting. Got an app for that? Well Frog kinda does, its called tvChatter (coming soon):

You don’t have to configure your favorite tweet app with filters for # tags. Just find the show and follow the tweets, or tweet away! As people chatting about tv media becomes more and more real time, it actually shapes and changes what we know about people using Twitter (remember when it was a social microblogging platform like what two years ago?).

So is this all becoming that all too intrusive computer interface for ‘learning about artwork’ that they give you on a hand held when you probably should just be looking at the painting? Or is everyone happy typing while watching? Naaman? You get a TV yet with actual channels?

The new social face of multimedia tagging.

I’ve never been too concerned with definitions—early in my graduate career I realized they were more often used for turf wars. Just as George Carlin fought to get a definition of what he could or couldn’t say, he showed us a description can be way more powerful. Lately, I’ve been describing quite a few things around people tweeting while watching TV or when at a concert. Currently, there are several great studies characterizing Twitter users. Less concerned with this, I was wondering, “if everyone watching the superbowl tweets what they think about whats happening, what does that say about the sporting event itself” (from a classic Multimedia perspective).

Using a sample of tweets captured from the first presidential debate, I began to investigate if conversationally, people behave the same way as they do when they watch TV. It turns out they do; my colleagues (Lyndon and Elizabeth) and myself were able to topically segment the the first presidential debate and identify the main people in the video, all by looking at a collection of tweets captured from the 90 minutes of the debate.

There are many gritty details (including the usage of Newton’s Method and Eigenvector Centrality) in the full paper to be presented at ACM MM’s first workshop on Social Media. Aside from methodology, we are suggesting there is more to media annotation than explicit tags on Facebook or Youtube. In fact, if Naaman tweets “I miss Paula #idol” while watching American Idol, he is leaving a comment/annotation on the media…despite there being no proper URI where Idol exists (yet!).

Recently, I was invited to speak at Stanford’s MediaX workshop on Metrics. At first, I was curious why I was there, I don’t think of metrics in my day to day life. I think about people and experience and stick figure drawings depicting the negotiation of meaning.

However, if we think about social behaviors and media (and now they relate to uncaptured events in the world): the methodological research becomes an exercise in metrics. What is happening? Is there a singular source event (or a plurality of events)? What do we measure? What does it mean to the source event?…I could go on. But you, gentile readers, can just read the paper or say hi at ACM MM in a few weeks or wait till I post more details about the work.

Social Media Definition (redefinition, that is)

Ayman, I know you’re sick of it by now, but I am revisiting a popular theme for this blog, “What is Social Media”. A definition of social media was attempted (by me) here, and I later added a note about a practical definition of social media in the context of teaching an interdisciplinary class on the topic.

So now, after teaching the first session of that class, let me try again. The following definition will try to broadly scope the topic as described in my Social Media class. But I also believe that this would make a good working definition of this widely-and-wildly-used phrase.

In this definition, I try to follow closely the original meaning of both “social” and “media”. Media is defined as:

the main means of mass communication (esp. television, radio, newspapers, and the Internet) regarded collectively. (Apple Dictionary)

And Social:

needing companionship and therefore best suited to living in communities “we are social beings as well as individuals.

These definitions are echoed in the following, although did not directly dictate it.  Social media, then, is any media that supports these two characteristics:

  • Posting of lasting content in public/semi public settings within an established service or system.
  • Visible and durable identity, published profile, and recognized contribution.

This definition would then include Facebook, Flickr, Delicious, MySpace, Yelp, Vimeo,, Twitter, Dogster, YouTube and their many, many likes.

The definition does not include purposely excludes old media services that allow for comments from users (no durable identity); Wikipedia (no “recognized contribution” that is easily associated with a user); or say, mobile-social applications (no posting of content => not a media!). The definition also does not newsgroups and discussion forums (no published profile, no expectation of lasting content). And it does not include communication services like IM and email that are not public, not even semi-public in nature.

Does this make sense?

On Media Multitaskers

(sidenote: I find that I now blog thoughts that are too long to fit in a tweet; so feel free to follow my tweets, or Ayman’s).

A recent article in the PNAS was quoted in quite a number of media outlets (Hindustan Times gave the Masters student responsible a PhD as well as professorship). From the article, Cognitive control in media multitaskers, by (the formidable team of) Eyal Ophir (get a Web page!), Cliff Nass and Anthony Wagner:

Results showed that heavy media multitaskers are more susceptible to interference from irrelevant environmental stimuli and from irrelevant representations in memory.

Heavy multimedia multitaskers (HMMs) are identified by a survey about media use, and compared to low multimedia multitaskers (one standard deviation over vs. under the mean of the index). The paper compared HMMs (not sure they are aware of the other meaning of the term) and LMMs on a number of tasks, finding that:

individuals who frequently use multiple media approach fundamental information processing activities differently than do those who consume multiple media streams much less frequently: their breadth-biased media consumption behavior is indeed mirrored by breadth-biased cognitive control.

In other words, those who multitask are not effective multitaskers – it’s the opposite. Of course, there are still outstanding questions:

  • Causality: what is the direction of influence? Do HMMs (I still find it hard to use this acronym) tend to breadth-biased consumption of media because of their distraction?
  • Index validation: how robust is the survey and metric created to capture the “media multitasking” index? Do survey participants’ self-reports actually attest to their real behavior, and does the survey really capture “multitasking” or something else? The authors note that Media multitasking as measured was correlated with total hours of media use — maybe that’s what was measured?
  • What other factors are in play? As the participants were all Stanford students, I do not expect major age, economic or education gaps; the also authors tested for differences in a number of dimensions (SAT scores, performance on a creativity task, ratings on extraversion, agreeableness, conscientiousness, neuroticism, and openness and others) and found no significant differences between HMMs (grrrr, acronym!) and LMMs. Does this cover it or is there any other factor that will help explain the differences?

In any case, interesting study — I am looking forward to the follow up work. And now, off to another media!

Twitter Evolves #nextstep

The good folks (partial list) at Twitter are doing their best to catch up with the emergent behaviors and ad-hoc constructs that rise from user innovation. These last couple of weeks we had announcement of official support for Retweet (mock from the Twitter blog below) as well as location data for individual posts.

Project retweet (from Twitter Blog)

While both are great, Twitter will only be ready to take over the web when they official adopt the next feature in line: support for adult-material spammers to add everybody as their follower at the same time.

Or, more importantly, hashtags.

A first-class support of hashtags will be the final nail in the coffin of Twitter taking over content everywhere (the Web, the world, the old media, TV, everything). Hashtags support would not only mean that a user can flag the topic of their post (#iran), saving a few characters on the way. Solid support for hashtags would mean that any user would be able to semantically tie their tweets to any type of object, virtual or real. Couple that with the flexibility of the Twitter system, and you have a platform where anyone can “attach a note” to anything, anywhere, anytime.

Examples? But of course. My tweet is about Society Coffee in Harlem. My tweet is about Sony Playstation III. It’s about the first episode of the Mad Men latest season. My tweet is about Rutgers SC&I. My tweet is about the web page of Rutgers SC&I. My tweet is about the New York Times article about Retweets. My tweet is about Ayman Shamma. My tweet is about Calexico.

Wait, how would that be different than just adding the hashtags in the text (e.g., #societycoffee)?. Well, Twitter people are smart. And they are friends of the good folks at Flickr. They will surely support Machine tags a-la Aaron‘s.

Machine tags will allow a much more robust (read: semantic) connection between the hashtag and the object discussed. I will still be saying #AymanShamma, but the system will store #facebook:user=111111 (or #twitter:user=22222). I will be saying “Calexico live in Barbi Tel Aviv” and the system will store #lastfm:event=33333 (Flickr’s machine tag now sports 1.2 million photos with a machine tag). Similarly, whether it’s a product name, a web page, a school name… a strong Twitter and client implementations can help users assign exact semantics (when they so desire) to any post.

Especially with location.

Context aware Twitter clients are a step away of being able to provide the users with the power to comment on anything, anywhere. I am pulling my iPhone out in a restaurant. My Twitter clients knows where I am, and gets IDs of nearby restaurants from Yelp. The client lets me select the restaurant I am in (or guesses it automatically based on the text and location). My post is now tied to the semantic object that is that restaurant (identified by Yelp ID, #yelp:biz=society-coffee-new-york-2) instead of just matching the text of the restaurant name (“Society Coffee” would not help much in matching and search tasks).

The Twitter API would surely allow other players to “read” all this content. Companies could show tweets about their products on the product page (or even ask users to tweet with #REI:productid=444444). If you are in a live event, a big screen can show all the content tagged #lastfm:event=555555 (which will be easy for any user to add to their post using their location- or calender-aware client). And more.

“If you liked this painting tweet #moma:paintingid=6666666”. We might see a lot more of these in the future. Twitter will bring on the object web. Just hash it out already!

p.s. Of course, our ZoneTag already did all these things (on Flickr) by 2007.

Teaching “Social Media”: Open for Suggestions!

The Naaman and Boase team are about to teach, for the first time ever (for us and at Rutgers), a “social media” class (informal announcement and silly photo here, Facebook group with some more information here). We are pretty excited about this opportunity (if I may speak for Jeff here) – I am looking forward for a very interesting semester.

But in the open teaching tradition I started last January, I am going to ask the one dear reader of this blog (it’s not Ayman, he just writes it) for input. What do you think a social media class should include? Try to think about it for a minute before looking at our tentative plan for the class, below. What did you hit that we didn’t?

Of course, two questions are immediately raised: 1) what is social media and 2) what is the target audience for the class. Let me start with the second, which is easier to answer. We target PhD and Masters students in various programs including Computer Science, Information Science and Communication (we even have a business school student registered). Letting both PhD and Masters students take the class means we need to balance theory/research and practical learnings that the Masters students can take with them to the workplace. Also, the interdisciplinary approach and audience means we will handle material from the social sciences, HCI and design, as well as computer science and information science topics.  One last thing to know: the students will form interdisciplinary teams to create/design a social media application (e.g. a Facebook app).

So, what is social media? Well, as you can see below it is the topic of the first session, so I am not going to give the full story here. In short, we see social media in a new (an emerging) Information and Communication Technology (ICT) that allows people to communicate in a public or semi-public manner, with emphasis on the personal identity of contributors and social connections. I will keep it short here so just a few positive and negative examples: Facebook, Twitter, Flickr, MySpace are social media. Wikipedia, comments on New York Times articles, Instant Messenger and Newsgroups are NOT social media. Let’s argue about that definition later…

So, what would you teach that’s important to understand this emerging ICT? What are the key readings that are not to be missed? Here’s what we have for now, without the readings. Feel free to suggest your favorite reading on each topic, as well!

  • What is Social Media: introduction, definition and examples.
  • Communities and social networks: Concept of communities, offline and online, how this concept is shifting; what are social networks (i.e. ties between people) and what to they enable.
  • Social network models and structure: online social networks, analysis of social networks, structure of networks, ego-centric network view, etc.
  • Open Platforms: “Web 2.0”, “the Web of data“, APIs – some idea of what can be built on top of existing social media applications.
  • HCI and Design: introduction to the design process with emphasis on Web and social applications.
  • Motivation and adoption: when do people adopt certain social media services, and why do they contribute to them? In other words, what is the motivation of people to join and stay active on social media sites?
  • Social media across cultures: a cross-cultural look at the social media phenomena.
  • Mobile-social and Social Media for Good: with a very special guest speaker!
  • Data Addicts: Data Collection, Analysis, and Visualization (for research or application purposes) of social media data.
  • Study Design and Data Analysis: introduction to research on social media services; how to design studies and analyze the data.
  • Social Information Design: an information-centric approach to social media; what are the different information factors in play (yes, this is where we talk about tagging).
  • Privacy, Legal Issues, Copyright, and IRB.
  • Economics of social media.

That’s it for now. As you can see, we plan jump from the theoretic, to the practical, to the research-y topics, hopefully making for a good mix. What did we leave out? What should we leave out? Your input is welcome, or as Dr. Boase would say, “we will try our darnedest to include suggestions, but may not be able to include all of them”.

Proposals and Innovation (Kudos to Google)

I think it was my colleague Michael Lesk that mentioned someone (I forget whom) performing a “back-of-an-envelope” (or was it “side-of-a-napkin”?) calculation, showing that the NSF proposal process results in negative gains to the research community (logic: number of hours writing proposals by researchers everywhere vs. the number of hours of work eventually funded).

I am sure Michael, a former NSF director, doesn’t completely believe in that, but I also believe that this calculation overlooked a very important value-generating factor: idea diffusion from the academic to the private sector.

You know how sometimes you have a great idea or insight, only to discover that somebody else already had that idea? Even worse, you know how this happens and you realize that you actually read that paper where the idea was described but forgot all about it? (yes, Ayman, this happens to people my age, you’ll see when you get there).

My thesis is that while not all NSF proposals get funded, all of them are reviewed and evaluated by panels that include many researchers in  companies. The ideas in these proposals stick in the minds of the readers, and could very well be unconsciously adapted or used later (I am not saying anyone is out to steal the academics’ ideas!).

This is why I find Google’s Faculty Research Awards an entirely good idea, both for the faculty and for Google. In particular, the open nature of the awards program (not specifically tied to current Google products or data, for one) is a key feature. Yes, G will get a lot of submissions and will spend valuable time reviewing each and every one of them. Obviously, the funded proposals will benefit Google as they create a direct link between the researcher and a Google person that will learn about the findings. But even the mere act of having all the world’s researchers sending Google their ideas and suggestions is bound to leave some trace in Googler’s minds. Google thus increases the funnel of innovation and ideation (I hate that word) to collect input way beyond its own engineers and employees.

Now, if they end up supporting my proposal, that would prove that they are even more brilliant!

Understanding the Creative Conversation

In October, I’m running a workshop (with Dan and Kurt) at Creativity and Cognition 2009. This workshop builds on the workshop I ran with Ryan two years ago at C&C07. In 2007, we had a great collection of artists, dancers, musicians, educators, ischoolers, and cs folk. This year, I hope we can further strengthen our focus:

This workshop is aimed at describing the elusive creative process: addressing models of creative practice, from art to craft, from dance to education. In particular, we wish to discuss creative models that are conversational: connect the creator and the consumer via the creative act or artifact. We invite researchers and practitioners from any domain to enter into the conversation about the design and process of the creative act.

Do check out the call for participation and we hope to see you Berkeley.

PS: I’ve located Naaman and have begun a creative conversation with him that I’m sure you’ll soon read about.


I am not sure I have a lot to say about this presentation, delivered as a keynote at the Symposium on Spatial and Temporal Databases (SSTD ’09) and embedded below, except that I think the visual design of my slides is somewhat improved.

Oh, and also that I tried to introduce the opportunity of social media data to the smart people in the SSTD community. Especially as Twitter is rumored to add location data, we are about to witness a significant new information system with social, spatial and temporal data all at the same time and in a never-before-seen scale. The opportunities are, as you might guess, endless.