Sunday, May 27, 2007

Collaboration: The Key to Success (repost from 2005-04-15)

Sometimes it is difficult to know how to go about leveraging an open source application like OFBiz in your company. This is especially difficult when people involved are not used to working on projects with larger teams and overlapping objectives. There are some misunderstandings and very tempting ways of doing things that everyone should be careful of, and it really comes down to effective collaboration....

One misunderstanding I see frequently is concern over occasional bugs that are discovered or changes being made to things you are basing something on, or in general not trusting the code or not liking the fact that additional work is required in some cases to make your own stuff continue to work based on changes, or be delayed because of a bug in something you need. Based on my experiences digging into these concerns, they are phantom concerns and is not the real issue. The real issue is the thought that there is a need for some sort of a stable set of code to work from, or that a strict isolation from the open source project is desired.

BE CAREFUL, THIS IS DANGEROUS! I say that in all caps because I really mean it. The fact of the matter is that the company is engaging in a development project, or more a customization project. You have a high level of collaboration happening within any company or group doing an OFBiz-based project, with sometimes one or two or sometimes dozens of people who will be making and committing changes to the company source repository. The aspect of this that I am talking about that is dangerous is the idea the you want to take a "snapshot" or somehow theoretical perfect set of code to use as a basis for your efforts. Unfortunately this theoretical perfect starting point doesn't exist, not in OFBiz, and not in any other piece of software open source or commercial.

A better way to look at this is that each person is working in collaboration with a number of other people, all of whom are working on projects that in one way or another move the software in the desired direction. The key thing to realize when working with an open source project like OFBiz is that there are many more resources than just those inside the company that are helping with a larger effort. Some of that larger effort (ie the set of all efforts going into OFBiz) is of interest to the company, and some of it is not. From an enlightened point of view, a LOT of things that might at first seem unimportant to the company and its specific project really are important and will over time save the company a lot of money, and the people involved with the project a lot of effort.

In other words, the fact is that OFBiz is a very large piece of software with many features and it is only because of those features that this specific project for this specific company can be a success for the effort being put into it. Software like OFBiz requires a LOT of work for maintenance and even though the company is putting a lot of resources into the effort, it is almost universally not near enough to maintain the entire set of software, fix bugs, create new solutions to problems that come up, modernize old code to make customization more efficient, and so on. Various groups have taken the "snapshot" approach and eventually they lose out on the thousands of hours of effort that result in improvements in OFBiz every single month.

The downside to this is that other people will work on new features that go into the open source project but that the company doesn't care about (not at that moment anyway), and in some cases those can conflict with what the company is doing. In most cases evaluation of the conflict results in a significant improvement in what the company ends up doing. But still, yes, there is some flexibility and reaction and conflict resolution that is required in the company to handle the side effects of what others in the community are doing. Still this cost is usually not even in the same order of magnitude (and often 2 orders of magnitude, in other words 100 times) separated from the effort that would be required to reproduce in the company everything that is going into the open source project that is of value to the company.

Some strategies to consider... Unless there is a specific reason for it, like getting a specific bug fixed or getting a new feature in place, it is best to only "sync up" with the open source project once every week, or every other week even. This is a very common tactic used in large development teams that are segmented into different teams with different purposes. In order for each team to move fluidly, they don't necessarily stay in sync continuously, but instead work in their smaller groups and continuously stay in sync there, and then frequently (but not continuously) combine all efforts into a single code base. Once the "feature freeze" happens (before the testing phase in preparation for a deployment) this process changes and typically only bug fixes are introduced, and everything is continuous.

There is a good book that I found recently on this topic that explains how to facilitate these sorts of things using Subversion. It is called: "Pragmatic Version Control, Using Subversion" by Mike Mason. There is also a CVS version of this book, but note that because things in CVS are less flexible (like not being able to move files and directories, for example), the approaches for CVS are much more difficult.

In general the key to success for any large project is: collaboration. When working with open source software effective collaboration is even more important, and at the same time the open source way of creating and maintaining software is a major facilitator of collaboration.

The trick is for people to recognize some of these key concepts and then embrace collaboration, and not fight it or fear it.

Some "Rules" for Collaboration that might be good to present and discuss and frequently remind people of in an large project:

  1. understand that the effort is much larger than any one person can accomplish
  2. recognize the value of what other people are doing in the larger scope of the project
  3. recognize that other efforts may at times require modification of the same files that you are modifying, and therefore sometimes introduce conflicts that will require your effort to resolve, and all for the better good and overall progress
  4. accept that you and everyone else WILL make mistakes along the way, and give people a chance to correct their own mistakes by letting them know about the problem you found; note that sometimes the problem is caused by multiple people, and multiple people will have to work together to solve them

I hope that I am speaking to the real issue, and the real fears and anxieties that people are facing. From feedback I hear every day this is a VERY common fear and misunderstanding.