Editor's Note: This is the fourth installment of The Decoder, a column that's part of the Living with Data series exploring how our online data is tracked, collected and used. Do you have questions about how your personal data is being used? Curious to learn more about your daily encounters with algorithms? Email The Decoder at email@example.com or submit your question via the form here. Screen shots and links are helpful clues!
Jean Yang went on a big trip through Europe this summer, from Edinburgh, Scotland, to Dubrovnik, Croatia, to Oslo, Norway, and back. Like a good tourist, she took pictures on her phone, an Android, throughout her trip. When she returned home, she found a surprise package in her Google+ notifications: a neatly collated, summarized, annotated digital scrapbook titled “Trip.”
Jean shared the album with me with this message:
This isn’t the first time Jean noticed Google doing funny things with her photos. She mentioned she has seen Google animate sets of images when she takes a lot of pictures in a row. She described one that involved falling snow to match the scenery. When she takes 10 images in a row, she’s not trying to create a GIF; she’s just trying to get things to look right.
Auto Awesome — falling snowflakes and all — is on by default for Android devices. It’s Google’s automated image and video editing algorithm. It’s pretty cool, but it can be pretty finicky. Google teases, “You might get copies of them made into things like short animations or wide-screen panoramas.” Google then notifies users once a picture is made “awesome.” The images are private until you decide to share them.
Apple has similarly started to organize photos in iOS devices by location and time stamp as Collections and Moments and offering smart editing tools. And it’s not so different from what Facebook introduced with Look Back, which summarizes important anniversaries.
For Jean, the auto-magically curated packages feels different from a helpfully organized, glanceable summary. It feels a little more like when your mom cleans your room for you. And in fact, Google likens sifting through your 437 photos to taking care of your piles of dirty laundry after you return home.
Jean admitted the feature was useful, if a little creepy. She likes to take notes about the places she visits on her travels, especially to share recommendations with friends later. When Google captures those automatically, it could be pretty useful.
Google wanted to develop a personal news feed of moments from all the data it has about our activities, and it ended up building a “personal biographer” into every smartphone. Google, of course, has a lot of data about what we’re doing, and it wanted to turn it into something useful by solving a specific problem — in this case, taking a lot of pictures and not doing much else with them. Stories fills a hole in apps that help document adventures. Alexis Madrigal offers an interesting look into technical challenge of turning data into stories like this.
By Google’s count, Jean’s trip consisted of 24 days and 117 “moments.” But how does Google determine which images are the best? And why did it choose an image with the most faces and therefore the most friends as the cover image for Jean’s “Trip?”
Auto Awesome elevates “best” photos based on exposure settings, cutting out duplicates and prioritizing clarity and sharpness. It also features people and landmarks.
Auto Awesome also takes multiple pictures and captures everyone smiling, essentially doing the Photoshopping for you. Sure, it may be great for capturing a group shot, but did that moment ever really happen? Reminds me of some deep philosophical questions about the nature of photography…
Of course, Stories’ scrapbooking algorithm isn’t the be-all, end-all of photo editing and curation. You can go in and edit a Story to add captions or change locations. You can also add pictures from other cameras.
So what’s in it for Google? Jean found this album in her Google+ notifications, so it might be another way to keep us coming back for more. For now, it’s just a summary of your trip, but we have to remember Google is at heart an advertising company. How soon before the sponsored restaurant details show up in the Stories that you share with friends?
Do you have questions about how your personal data is being used? Curious to learn more about your daily encounters with algorithms? Email The Decoder at firstname.lastname@example.org or submit your question via the form here. Screen shots and links are helpful clues!