over 9 years on 2009-07-30


-- this is a fully nyc story -- during internet week new york i collected all of the location check-in data from foursquare from all of my foursquare 'friends' in new york.  i used a simple little generalized email -> csv converter (http://csvemail.com) to turn the email based location 'checkins' into a mini-dataset and used some old skills to pivot away in excel.  it took me a while to get it out, but when finished i was able to retell the true story of the week of events straight of the gps feed (post below - covered by @caro).

it was fun and interesting to put together, and everyone loves some good data porn -- the problem was that it was annoying to put together, a slow process, and limited to the people in nyc that i was 'friends' with.

a few weeks later i was having breakfast with jon steinberg, who had created 'dropboxee' a wonderful drop.io boxee application (so you can watch/listen/view all your media directly off a drop on your tv in boxee)...  we started talking and decided that it would be fun to turn the experiments i had played with during internet week new york into a mini application using the newly alphaed foursquare api.  

1.  allow people to 'sign-up' to the app
2.  poll for their location checkins (and all the location checkins of their 'friends') on a cron job
3.  drop the duplicates
4.  create and publish a dataset (maps and leaderboard) of near-realtime 'hotspots' in each city - by hour, day, week, month

jon went off with william piel - and thus was born socialgreat.com - an application which uses 'social' to tell you what is 'great' in near real-time in any city (but starting with nyc).  think of it as a real-time zagats based on where people actually go (or, at least where they choose to check in) vs. star ratings.

the foursquare guys started calling it 'trending for foursquare' -- or 'trending for locations' -- that is a really solid way to describe it.  just like trending topics on twitter, this isn't about necessarily what your 'friends' are up to, it is a sledgehammer for the conversation/universe of data.  for a while now we have been watching as people nightly shuffle in and out of 'tom and jerry's' -- wondered what cool event is happening when we see some random bar shoot to the top of the list -- wondered why we wen't invited to what appeared to be a rocking party from a checkin standpoint at a mets game, etc.... and it only gets better as the number of people we can watch grows (now over 2000) -- i almost dropped in last night at some place called the center 208 w 13th st just because apparently a lot of people suddenly showed up there, must have been something cool

the app will continue to evolve - more cities - more visualizations - and more analytics, as foursquare and other location based data sets (and our data set as a result) expand to support the granularity -- ... i will keep dashing off emails now and again asking for new visualizations on the data (some really awesome and useful ones coming soon) to truly uncover when and where it is best to hang out... hopefully team bain nyc is satisfied with the data education they bestowed, and some new ways to apply it to new data-sets in new contexts.  

of course - given the general style i blog in -- it is worth noting that, this is just a social tool, collecting seeds of data sprinkled here or there for social enjoyment and perhaps adding something to the world -- the real implications of location data run far far deeper, with datasets the size and complexity of which are truly mind boggling...  that is when you get into the world of a different nyc based powerhouse of the future... sense networks -- when you want to talk about what this really all means, and how to deal with a location future -- they are the game in town.  nyc, top to bottom, ftw on the location front.

p.s.  infact sanfransisco is now live -- if you live there, check it out http://socialgreat.com/sf

original swl blogposts and letters 2007-2010