You’re probably familiar with the facial recognition technology in Picasa and iPhoto. It’s neat stuff; just pop in your SD card and all the smiling members of your grandpa’s barbershop quartet are automatically recognized and tagged in each photo. Sometimes the algorithm is a little unsure and asks you (the expert) for confirmation, but for the most part it works pretty well.
But what about video?
Imagine that your son is turning 10 in a few days. Wouldn’t it be cool to show a video montage of him growing up at the birthday party? Yeah, that’d be awesome. It’d start with a video of bringing him home, then it’d cut to him doing something cute at age 1, and then at 2 there’s that hilarious video of him plopping a bucket on his head, yeah you definitely need that one, then Disneyland at 3, etc.
But, what year was that bucket video? 1995? Yeah, that’s right, it was during the Corn Festival, and that was in June. Okay, so there’s 13 and a half hours of video from June in 1995. When was it. The middle of the month. Alright, let’s pick this one and fast forward. Nope, next one. There it is, alright, one clip down. Now when was Disneyland? It was either July or August…
That’s what you would have to do now. Many, many hours later you’d probably have something that’ll have the group going “awww,” but boy was it tiring. Your index finger cramped up like, twice from all that clicking. But with facial recognition in video it’d be easy:
From Keren Master of Microsoft Innovation Labs:
HomeVideoX offers a new unique experience that allows you to browse the content based on the scenes specific people appear in. With HomeVideoX you can easily find the scenes that interest you and export them to a movie editing tool such as Windows Live Movie Maker.
Straight to Movie Maker? Yes!
This is what we all need (not including the Movie Maker part). With a phone in nearly every person’s pocket nowadays, you can be sure that there are plenty of computers out there with a whole bunch of unorganized videos. Maybe some people have their videos organized in an elaborate folder system, but that still isn’t effective enough as we role-played above. Searching through video needs to be simple, easy, and fun, and Microsoft’s Israel R&D Center is the closest to making that happen.
Alright, video-watching time. Check out these two videos of Microsoft’s OneVision Project.
Incredible right?
Face recognition in video is an emerging technology that will have great impact on user experience in fields such as television, gaming, and communication. In the near future, a television or an Xbox will be able to recognize people in the living room, home video will be annotated automatically and become searchable, and TV viewers will be able to get information about an unfamiliar actor, athlete, or singer just by pointing to the person on the screen.
iLabs’ OneVision team presents a new innovative Machine Vision algorithm to provide face detection, recognition and tracking in both still images and video.
Just a few months ago I was pondering whether or not it was technically possible to tag somebody in a video. The way I figured, if Picasa was able to detect my face in a photo I took of a video playing on my computer, then by golly somebody should be making a program to recognize faces in video. But I wasn’t quite sure if scanning every frame would be feasible, after all, Picasa’s facial scanning takes a while to complete.
Well, Microsoft has apparently thought about everything:
OneVision Video Recognizer automatically indexes the entire movie frame by frame… The recognizer detects faces throughout the video and assigns a unique ID to all the sequences that feature the same person. Once tagged using a simple tagging tool, the face recognizer automatically identifies the tagged people at any point in the video, even if more than one person is in the frame, scenes have changed or people are not directly facing the camera…
Home Video X analyzes your home videos by detecting and clustering faces and providing a simple tool for quick tagging. Once the movies are tagged, you can browse them and identify interesting clips based on who appears in each scene. You can select a set of movies and see who appears in these movies or select a set of people and automatically get the list of movies and scenes in which they appear. Home Video X supports both individual and group searches.
I’m psyched for this, I can’t wait. When this is implemented, the trifecta of digital content (text, photos, and videos) will be completely searchable and index-able. It’ll be a digital neat-freak’s heaven. Make it happen Microsoft, Apple, Google, whoever; complete the trifecta!