On Collaboration

Collaboration, collaboration - seems like such a good idea. "All of us are smarter than one of us" as the saying goes (or in a very perceptive counterstrike I ran across "One of us is smarter than all of us"). Is it the wisdom of the crowd or mob rule? Exciting emergent behavior from a group of like-minded folks or the tragedy of the commons? Well, I could go on like that forever but here's my take as it deals with tech projects and libraries.

Collaborative efforts are largely a good idea - frequently to accomplish something big and sustainable its unreasonable to expect one person to do it all. The question is how do you collaborate effectively and at what points of the project? Clearly the brainstorming phase is a good one (was going to say no-brainer but that would be unfortunate). Clearly its good to have people able to divide up the work, evenly if possible but at least according to their talents and interests if not. My experience is that collaboration fails largely in the design and management areas (and, as a result, sometimes in implementation).

Let's start with the idea of the structure of the collaboration. It seems there are some really neat things out there coming online that start with little or no structure - literally Two Guys (or Gals) and a Garage. While this is an exception to the rule it shows it is possible. I'd put this on the opposite end of the spectrum from the Major Organization or University initiative that started somewhere high in the adminisphere and slowly made its way down to the parties involved in actual implementation like a brain-trickle (think of a Rube Goldberg device in the Amazon rain forest collecting the rain if you need a visual image). Somewhere in between you have the collaboration of equals in a volunteer or non-profit project - probably the cast of characters is wider and fits somewhere between the "let's just do it" and the "this should get done, let's appoint a task force to study the possibility of a committee charged with determining the feasibility of doing it".

The small, nimble project can do a lot and do it quickly and has great fidelity from idea to implementation because its such a small and presumably like-minded group (not to mention voluntary - the Two Guys both want to do this and believe in the project). If the project doesn't go anywhere the Two Guys will be unhappy (may even lose their garage!) but its probably not the end of the world. Its like a garage band - a few become hits, the rest wind up working day jobs. Everyone takes equal responsibility and everyone basks in the rewards (or punishments) of their results.

The large, adminisphere based project usually takes a very long time to produce an actual product, but in all fairness they also have a more realistic chance of "scaling up" to a national or global capacity should their product catch on (it may also be a product that integrates with other products they already have). On the path from idea to task force to committee to implementation there are delays that are unavoidable due to logistics (how much time is spent in meetings? How much time is spent planning and traveling to and from the meetings? How complicated is it to get capital and labor when you have to compete with all the other projects out there trying to do the same?). If your project involves technology you already have a strike against you because the Two Guys can probably get their product to market faster. On the other hand you likely have a large advertising budget and existing promotional or sales and technical support staff. If in an educational institution just substitute the word "curriculum" for "sales" (they both serve similar functions - getting the "right stuff to the right people in a timely and coordinated fashion" - and require similar administrative overhead; the main difference being a captive or non-captive audience).

The volunteer or non-profit project is the most interesting, and dear to my heart. It also suffers from the most challenges. Its usually large enough to have administrative overhead like a business or university. It usually has a variety of people with different points of view which is good in the brainstorming and implementation phases, but requires more administrative or management time than Two Guys would. Unlike the business or university it usually does not have funding or any sort of advocate to take them under its wing so to get things done it either needs to have extremely motivated participants (who stay motivated over time) or a way to get external funding. This could be grants (which necessitate a whole painful new level of administrative overhead, hoop jumping and slowdown) or donations (which then require a financial infrastructure and usually an attendant administrative one to make the money gathering legal - translation: bylaws and legal status). In short, these projects have none of the advantages of bigness (existing infrastructure and talent) or smallness (nimbleness and lack of necessity for a lot of infrastructure).

The good folks at 37Signals (developers of fine and dazzlingly useful software products of limited scope - they don't do a lot but they do what they do very well) put out a book called Getting Real which really turns the thinking of much of the "big" project people on its head. In fairness to the big people, many of the suggestions are probably not realistic for, say, a corporate or university environment. But I think the book is essential reading for anyone involved in technical projects and, who knows, it could be the savior of the "middle way" volunteer projects. They focus on being nimble and agile, not adding unnecessary stuff to your products or your workflow and management structure and generally dispensing with a lot of what I'd call "inertial management nonsense". Thankfully they're also not going the Stephen Covey route and trying to pass this off as the "next big thing" in management or living your life or the like - its just things that worked for them, and may well work for you if you'd give them a chance.

My take on where bigness vs. smallness works is this; in general larger groups are good for brainstorming, evaluating the project closer to rollout, and doing the maintenance or updating work, but in the planning and design phases its often better to have a very small group to actually carry that work out. Design by committee is usually an ugly thing, but if the larger project people can work out the ideas, hand it off to the smaller design group, then they can in turn give it back to the larger group for implementation things may work very well indeed. All of Us can, indeed, be smarter than one of us, but All of Us can also suffer from diffusion of responsibility, groupthink, laziness and a tyranny of the Big Mouthed. One of Us will have trouble pulling off a project of any large size, and for any length of time, and so should seek out All of Us for input and evaluation. They should then go off and do the work, adding their own special magic that doesn't come out of an assembly line, and deliver it back to Us to bring to the world.

Projects of all sizes would also do well to look into newer, distributed collaboration tools which are starting to become more prevalent and easier to use (and I don't mean Microsoft Project). 37signals makes really neat hosted products like Writeboard (shared note space) and Basecamp (entire shared project management space). There are also cheap or free tools like blogs and wikis and social networking sites like del.icio.us which, with the proper technical support and training, can supplement projects in ways email lists can't. If all your people share a network you can have a shared network file space, or if not its trivial to set up a basic project intranet on a website. Again, having a small group of people maintain something like this can benefit a much larger group with a minimum of effort (and the more your people get used to using such tools the easier it gets - remember, most of us didn't used to have email and the web browser). Using shared tools like this also give far more people the ability to participate meaningfully in the project and often serve as a useful way to collegially "keep each others' feet to the fire" along the way - too often things locked up in committee or board meetings drop off other people's radar screens and the whole project spends way too much time playing catchup. Its a fast track to discouragement if decisions take forever to make and nobody knows how far along things are in process.

Shorter this column: Sometimes a collaboration takes place in a large environment, where much of the time is wasted on administrative overhead and worthless meetings. As things get closer to the implementation phase perhaps the collaboration can be saved if its going off the tracks, though if you could get all the people impacted by it in the loop earlier that would be better. Sometimes a collaboration is just a few people who work well together and come up with something amazing on their own. Often these collaborations won't scale but that's not always the case, and sometimes they can beat the pants off a better funded and more "eminent" competitor. We could all learn a lot from these people. Sometimes a collaboration is more in between - a group of professionals or dedicated people who know they need a larger infrastructure to spread out the work and survive long term but aren't supported by a large company or organization. These people could *really* learn a lot from the Two Guys in a Garage approach. No matter what the size of your project you'd probably well to keep the management and design functions down to a small group of people who work well together - you can handle the larger, unwieldy parts (and benefit from them) if you keep them to the brainstorming or initial input phase and the implementation or shaking out the bugs evaluative phase once the product has been largely built. You can also use a variety of new collaboration tools (many of which are cheap or free) to cut through confusion and save wasted time in meetings going over documentation. Repeat as necessary until everyone reaches nirvana.