Archive for September 2014

The 1-hop neighbours problem

The problem of computing number of friends connecting two individuals forms the heart of people recommendation engines. This problem is easy to express in small data contexts (e.g. using SQL) but are notoriously hard to solve with good performance on massive, real-world networks. We discuss why.

Calling out the big data scientist

“Data science” is a popular term and one in the ascendancy in Gartner’s Hype Cycle for Emerging Technologies 2014. It has multiple meanings based on whom you ask. One way to deal with subjective interpretations is to crowdsource the answer and pick the popular interpretations, provided there is enough data. Recently, a data scientist (who else?) at LinkedIn attempted to define the term “data scientist” using data from profiles of people that have the phrase “data scientist” across