The Power of Metadata: Deepak Jagdish and Daniel Smilkov at TEDxCambridge 2013 (Transcript)

Deepak: What you see here is a small slice of my life. It’s a picture of five years of emails that I have sent to my contacts, from 2008 to today. Each bar represents six months. Clearly, something happened in the second half of 2010. Something significantly changed in my communication pattern.

A friend of mine, whom I shared an office with at the lab, happened to see this. And he was quite concerned and curious for me. He knew I had just started a job around that time, so I wasn’t on a vacation. And that led him to ask a very direct question: “Hey Deepak, did you go through a very difficult personal situation?” My friend was right. That was a time when an important personal relationship had ended for me. Following which my circle of friends changed, and I went through a long period of self-reflection.

Now what’s odd is that my friend knew nothing of my personal history for him to be asking this question. And also the only parameter that was used to construct this picture, was the date field in my emails. The date field is part of email metadata. And metadata is what we want to talk to you about today.

Daniel: So what exactly is metadata? Metadata is data generated from the interactions you have with other people in organizations, as you use technology. In a personal context, it’s about who you call on the phone and when. It is about the time and location where you have swiped your credit card. It is about the recipients and the time of every single email you have exchanged.

So, Deepak and I are graduate students at the MIT Media Lab. And since we work with data, a natural question for us was: what can we learn from it? We realize that understanding and appreciating metadata is difficult for most people, because the interfaces we use to interact with our data are shallow and repetitive. Take email clients as an example.

For the last several decades, our emails have been presented to us as a time ordered list. Every single day, we get the same view with just a new set of emails replacing old ones. But the number of emails that we’ve sent and received over many years is way more. And as we send and receive these thousands of emails, we leave behind unique digital traces. So we realized what we lack are tools that can help us revisit and learn from our own digital trail.

Deepak: Speaking of trails, let’s go on a road trip together. And imagine that on this road trip, this is all that we get to see. That’s a view of the road right beneath us. Now, there are other ways of looking at the road trip too. For example, that view. But there’s another one, like this one.

Now if I would ask you which one you prefer, most of us would say the second and third ones. Why ? Because they provide us with perspective and context. They tell us where we come from, where we are at the moment, and where we’re headed to. And a digital trail like the one Daniel was talking about is no different. And frustrated by getting to see only the road in the context of emails, we took a small step towards solving this problem. We created a tool called Immersion, that combines, analyzes and visualizes your email metadata.

You see, Immersion only looks at information above the subject line of emails. Which means it looks at only the From, To, Cc, and the TimeStamp, never touching the subject or the body content of emails. When we built this project, one of the philosophies we had was to center it around people and the social links that are found around people; not basing it on ordering of TimeStamp messages.

And privacy was very important to us. So any user of Immersion has the freedom to delete any of their metadata that is collected by that tool. And Immersion has the power to transform the raw metadata that you see in emails into a visual form, that can reveal much more than what you see today through your clients. We’re going to show you what Immersion looks like right now.

Daniel: And we’re going to show you how it looks like using just my metadata. Immersion represents my contacts as circles. And simply by counting the number of emails exchanged with each person, it sizes the circles accordingly. Now, email conversations between multiple people are represented as lines between circles. So seeing Deepak and César connected by a line, means that the three of us had conversations as a group.

Now let’s focus on my relationship with Deepak. When I click on his circle, I can immediately see the people that Deepak and I have been in contact with together. And the thicker the line between Deepak and another person, the more emails were exchanged between the three of us. The person that stands out in my relationship with Deepak is our advisor, César, for obvious reasons.

Another thing is that Immersion can show me how my relationship with Deepak evolved over time. If you take a look at the histogram on the right, it’s very easy to see when I first met Depak, when we started working together, but also when we became very close, when we launched the tool. It was when things got crazy — that tall bar over there.

One other thing is that, if I click on César now, and if I see who he has introduced me to, I can actually see who César introduced me to. And that’s just by metadata. And I can see that there are plenty of people in that list, so he’s definitely helped me expand my network. I learned something there. What is also apparent, is that this network spatially organizes the different groups of people.

Pages: 1 | 2 | Single Page View

By Pangambam S

I have been a Transcriber and Editor in the transcription industry for the past 15 years. Now I transcribe and edit at If you have any questions or suggestions, please do let me know. And please do share this post if you liked it and help you in any way.