A Clustering Quandary

by New Idiom


I am facing a clustering quandary. To help you understand why I am “quandaring” here is a brief overview of clustering, according to ID.

Part of the process of systematic design thinking is developing a set of insights and then grouping them according to their likeness. According to popular opinion with only a few ideas, say 30 or less, it is easy to group them manually, back-of-the-envelope style.

However, after that, it becomes much more difficult to group ideas, because remembering all of the nuances of the ideas becomes unmanageable. To overcome the difficulties of managing the relationships between ideas, you can use tools, such as the insight matrix to help manage the relationships. The insight matrix depends on human input to rate the strength of the relationship, regardless of what that relationship is. Then, based on that human rating of separate and independent relationships, the excel plugin groups similar ideas, or ideas that share similar relationships to other ideas, to form clusters of ideas.

Beyond that method for grouping is Chuck Owen’s Relatn software tool. This tool also depends on humans rating relationships between ideas, but its controls are more nuanced than the controls of the insight matrix.

Quandary begins: A short time ago I was part of a team that grouped over 100 ideas manually, without the use of any tool beyond our own memories and the ID-loved Google spreadsheets (any other spreadsheet would have done just as well). We managed to group our 100+ ideas in around 6 hours, a process that would otherwise have taken us more than 41 hours (you do the math: 10000 individual decisions to make, at 15 seconds per decision) using the Insight Matrix or Relatn software. We ended up with 20 or so very manageable groups from which we were able to derive design principles and concepts in their respective orders.

Ultimately, what was lost by our using only our memories and a spreadsheet (with no plugins) to manage the groups? First, one might argue that we may have lost some nuances of relationships. We probably did. It is quite plausible that because we did not look at one idea 99 different ways that we did not really leave room for surprising relationships to appear.

Second, we missed out on a lot of discussion generated from the juxtaposition of two seemingly unrelated ideas. In past projects this discussion has been invaluable in tapping into each others’ perspectives and experience

Considering what we might have missed makes me contemplate actually going through the painful and time-consuming scoring process based on these same ideas. What mysterious clusters might have grown out of rating the strength of 10,000 different relationships? What new and surprising perspectives might we have gained through a week of team discussion?

I guess I will never know. But how valuable is a work week spent scoring relationships as compared to one day scoring relationships and 4 days…doing other things?