Tuesday, June 23, 2009

Open Source Community Collaboration Best Practices

WARNING: This is a long post, approaching a dissertation on the topic. I recommend that you glance over the headings, and then go get some tea and cakes, coffee and donuts, or whatever it is you enjoy consuming casually while having your brain tickled.

The Best of Practices

The title to this article falls a little bit short of accurate... I actually only want to talk about a single best practice that seems to consistently determine whether or not an interaction with others in the community will be successful or not. The idea of this give and take is really simple, and goes back to the basic definition of collaboration:

1. the action of working with someone to produce or create something

2. traitorous cooperation with an enemy

Both parts of this definition are interesting, because the practice that leads to success in community collaboration is one that aligns well with #1 and avoids any appearance of #2. On either side of the definition it comes down to who you choose to collaborate with, and that's the most important thing to keep in mind in an open source project: you are collaborating with people and expecting to both give and take as part of your interactions with them.

If you want other people to collaborate with you, first you need to collaborate with them. If you are hoping for some benefit of the collaboration you first need to set the stage for the collaboration and invite others to collaborate by starting first and giving what you have, then invite others to get involved. The important thing to keep in mind is that collaboration implies a two-way street and if you try to make it one way by not trying to collaborate with others, but still expecting them to collaborate with you, then you will most likely be disappointed by either no collaboration at all, or a good-will attempt by someone else to collaborate with you that will fail because no stage was set for the collaboration and they will likely help in a way you don't need.

Now a quick "anti-pattern" to explore what this implies... you may be wondering: "how can I be sure that other people will collaborate and I'm not just giving stuff away and will never get anything back?" That issue is certainly one that stands in the way of collaboration and basically represents a lack of confidence in the possibility of collaboration. The problem is that if you are unwilling to give it a chance and "cast your bread upon the waters" as it were, then there is not even a chance that you will be the fortunate recipient of collaboration in return.

If you can't give for the sake of fostering collaboration and good will, then community collaboration may not be for you. The best path may be to continue to collaborate through the financial and legal means that are the common operating mode of commercial software and result in a very inefficient form of collaboration (if you can even call it collaboration) that causes many of the problems that people and organizations face with the software they currently use. Unfortunately many commercial software organizations would consider open source collaboration to be firmly in the camp of definition #2 above.

One good concept to keep in mind when working with others is the "burden of proof." A good way to be generous with others and to make it clear to others that you are putting efforts into collaboration is to take upon yourself the burden of proof, and not ask others to take it on. When you do this you will communicate more clearly and others will tend to reciprocate and do their own research, puts their thoughts and effort into it, and make things flow back and forth more smoothly. What this means in daily practice is usually to just research things well and before proposing something or asking a question thoroughly present your research, including references and links where possible. In the Apache OFBiz Committer's Best Practices Guide this is referred to as "read before you write" and the burden of proof concept takes it one step further to sharing with others what you read so that you can collaborate on the writing side of things.

A Case for Collaboration

This reminds me of a quote from a movie scene (whose name I can't remember and google isn't helping): "Qui bono?" The line is from an old politician who has a clear set of priorities. I'm not referring to the "Qui bono" line from The Departed, though that one is more entertaining, and I guess almost equally as enlightening. Whatever the case, the point is: who benefits? I suppose the point from The Departed is also appropriate: who cares?

Commercial software organizations operate under a specific model and that is very understandable from their perspective with their investors and a wide variety of stakeholders that they need to make a profit and leverage whatever assets they have available in order to do so. Software is often considered such an asset, and treating it that way is extremely well supported by modern intellectual property laws. So, is that a good model or a bad model? In the abstract I think that's impossible to answer, and it depends on whose perspective you are looking at.

What about the perspective of the end-user? Which approach best benefits the end-user (individual or organization)? Is there a conflict between what benefits software producers and consumers? My opinion on this topic is that again the answers depend on who you ask and that nearly all large commercial software producers will answer in their favor, and some end-users may as well (perhaps because they are influenced by marketing messages) though for the most part they will answer differently based on actual experience with those large commercial software producers.

All of that introduces a lot of complexity, and it is possible to look at the simple flow of information under the assertion that the more end-users have influence over software the better it will meet their needs. Through open source collaboration end-users can very directly influence the direction of the software by participating in the project and collaborating with other contributors and users. And what about the crazy things that some companies do? The collaboration tends to be very effective for filtering out or refining ideas that aren't so great. On the other hand, sometimes the "crazy" ideas turn out to be a differentiator for the company and result in a huge benefit over competitors. With open source software they have the flexibility to implement such things anyway (and do so efficiently) and even keep those changes private if they desire (well, unless the software is GPL licensed, which I would assert is not a good license for business software that is meant to be customized).

On a more day-to-day note direct collaboration is powerful in many situations that users and developers run into. It allows new users to get feedback on what they are trying to do and either open the way for new functionality or get recommendations on how to better use existing functionality. On the other end of the spectrum, it allows experienced users and contributors to get feedback on both their requirements, designs and implementation approach and ultimately results in far better software that will generally meet current needs better and also be more likely to handle changing future needs. In other words, whether you are experienced or not direct community collaboration can very efficiently help you to resolve issues that you face, and exploit opportunities that arise.

In short the ultimate reason for direct collaboration, as opposed to collaboration through financial and legal means, is that it results in software that better meets the needs of end-users (more flexible), better protects the interests of end-users (more private), and is also less costly for end-users. That reason will always be true.

Right now there is another reason to consider more direct collaboration and that is that commercial software companies have become massive entities with a lot of influence over governments as well as end-user organizations... and this means that they are able to put their own interests above those of end-users and get their way as much as they want. Very large end-user organizations can sometimes stand up to them, but that is still a small minority, even in this modern world of business. Shifting that balance to something more in favour of end-user organizations will make non-software commerce more efficient and allow software to be more an enabling industry instead of a controlling one and ultimately enable general economic growth and stability.

For commercial software vendors there doesn't have to be a conflict with open source collaboration. They don't HAVE to consider that collaboration to fall under definition #2 above. It may require some adjustments, and ultimately by better serving their end-users they will offer better solutions and keep clients and customers for the long term. The downside is the sacrifice of short-term profits and the ability to boost profits by pushing clients to buy things at certain points in time, like scheduling releases and upgrades to boost revenue or introduce separate products instead of adding features to existing products and including them in free service packs. End-users don't like such practices and if they perceive a viable alternative they will leave... so why not change practices and find more profitable approaches that also better serve the end-user who ultimately pays all of our bills...

Collaboration on Software

Going beyond general collaboration there are certain things that must be done to more effectively collaborate when working specifically on software.

The most important is to collaborate on designs and not just on implementation, keeping in mind that to collaborate you need to start out by collaborating with others and giving them a chance to collaborate with you. In the software world that means don't just implement something and throw it out for everyone to try to digest and react to. Instead, write up what you want to do and give other people a chance to send their feedback. Depending on the complexity of what you are working on this could take from days to weeks.

So why do this? Won't it just delay your work? Consider that if you create a design on your own chances are you'll forget things or not be aware of requirements that are common and important to that design. If you leave those out and implement something then sooner or later what you implement will be changed, or thrown out and re-written. Why is that? Because important requirements were left out of the initial design, and those requirements won't just go away. If you're lucky the requirements will have a small impact and won't require significant changes to the design or the implementation. However, especially if you are not trying to make things as broadly applicable as possible then it is easy to ignore big things that will impact the design a lot, and then it may be impossible to just refactor and extend the implementation to support the new design.

How can we avoid this? The answer, as would be expected in an article about collaboration... is to collaborate in advance on requirements and get the design to a solid point before you start implementation. Your code will have a longer lifespan and almost always also be more useful to you and your clients.

Some Short How-To Scenarios

1. I want big feature XYZ to be developed and I would like to work with others on it to ensure the best possible design and so I don't have to implement all of it.

2. I want big feature ABC to be developed and I'm not a developer so I can't help with development.

3. I have a question and I'd like to get an answer through the community (I can't afford to pay someone to support me, or I think this is an issue that the community should address).

4. There is something that exists in the software that I think should work differently.

For all of these the pattern is the same. Start with research on what exists in OFBiz, the requirements driving what you want, what can already be done and what can't be done or that you can't figure out how to do, then present your findings and requests for help and collaboration that you would like to see happen.

Collaboration Anti-Patterns

In the software world one way to clearly not collaborate is to totally rewrite something instead of improving something that exists, especially if this involves no attempt to understand what already exists and make sure that the replacement is sufficiently better to be worth the change and also does everything that the old stuff did. To understand more of why this is bad for collaboration, and sometimes bad for design and implementation as well, consider some of the things that you would be implicitly doing:

1. throwing away the feedback (or potential feedback) from others on the mailing lists

2. throwing away the various fixes and improvements that people have put in

3. in doing #1 and #2 you are making it impossible for people to collaborate with you... as much as many people are trying when you take this approach you are not allowing people to do so

4. large scale re-writes when there are issues instead of addressing the issues usually results in things NEVER becoming refined (since the focus is on starting over instead of on refinement), and in addition leaves a very large wake of orphaned code

Final Summary

In short the best way to collaborate with others, especially in a volunteer situation like an open source project, is to start by investing in the direction you want and then giving that investment away and soliciting involvement from others (including a write up on what you've done and your vision of what it can/should be and perhaps even on how others might get involved). By "casting your bread upon the waters" in this way you open the way for others to join in and give back, to collaborate back with you. Others may not collaborate back in the way or on the timeline that you had in mind, so be patient on both aspects. Consider what others suggest and appreciate their suggestions and discuss them in the open. This will foster more collaboration and allow the end result to be far superior to what you are capable of on your own... no matter how talented and/or experienced you are.

This direct collaboration model is a powerful way of getting things done and maintaining them over long periods of time in inherently stable ways. It enables a high degree of innovation and eventually results in solutions that are eventually clean and efficient and effective. Again, be patient with it as things are generally not perfect on the first pass or during the height of the collaboration activity, but rather when the collaboration has taken its course.

Wednesday, June 10, 2009

A Time for Change

I have had the pleasure of working with a number of great people over the 8 years I've been involved with OFBiz. For the last 2.5 years I've been a part of a great and growing company, Hotwax Media, and it has been amazing to see the number of people involved with Apache OFBiz increase as well as the work done over the years (including excellent custom work and a large number of contributions back to the open source project).

Much of the world seems to be changing right now and more changes are coming as people adjust their priorities and move on to new opportunities. For me personally the time has certainly come to make some changes. I'm cutting back on both expenses and investments, and getting back to what I know best and enjoy doing most: independent consulting.

On Saturday June 6th I resigned from Hotwax Media, leaving behind a great group of people and even some cash on the table, and moving toward the many and growing opportunities for doing other OFBiz-related work. Apache OFBiz is a great solution in an economic environment (similar to the one it started in) where organizations are moving back to core values and focusing on accomplishing their core objectives. In some cases that means doing more with less, and in nearly all cases it means doing more of what distinguishes the organization, and doing that requires more control over the business and the software that helps automate and manage it.

While it's been great working with the people at Hotwax and with many clients through the company, I miss working more with others in the community and seeing more of what other service providers and end-users are doing. I'm truly looking forward to doing work similar to what I did a few years ago and working with a larger group of individuals and organizations.

Along with this I'm also looking forward to putting more time and effort into new objectives for Apache OFBiz, including adding collaboration on requirements and designs to the existing highly successful collaboration on implementation, and along with that creating some applications meant to be used out-of-the-box by specific types of organizations or individual end-users.

If you think I might be able to help you with something you have going, please look at my web site at http://www.dejc.com/ for contact information and more details on the types of services I am offering.

Tuesday, May 12, 2009

Apache OFBiz at OSCON 2009

O'Reilly OSCON 2009 is taking place in San Jose, CA, USA on the 20-24 July 2009. For details on the conference see:

Apache OFBiz was going to have a booth there but that has been cancelled because I was unable to line up enough resources to make it happen.

An OFBiz-related thing at the conference that is still happening is the presentation I will be giving on Thursday the 23rd at 2:35 PM. The title of the presentation is "Apache's Open For Business: Holistic Analysis and Design." For more details about the presentation please see the schedule here:

I look forward to seeing everyone there and enjoying some conversation and collaboration during a bit of the Northern California Summer.

OSCON 2009

Monday, April 13, 2009

Apache OFBiz Community Building Tour - Mid-western USA, April 2009

As a follow-on to the April release branch, and to help encourage community growth and participation for Apache OFBiz, I'm planning a tour of the Mid-western United States. As part of this tour I'd like to visit with individuals and organizations who are current and prospective users, contributors, and others involved with or interested in OFBiz.

In these visits I will be available to spend 1-3 hours with you and answer questions about Apache OFBiz and how you can use it most effectively. I would be happy to speak with individuals or as many people from your organization as you would like.

If you would like to better leverage opportunities to collaborate with others in the community or there are specific things you'd like to see in OFBiz, or be able to do with OFBiz, this is a great time to chat about it. Also if you'd like general business level or technical help with the software I'm happy to go over those sorts of things as well.

I am doing this as the PMC Chair of Apache Open For Business with the intent of helping grow the community and not in my role as an officer of Hotwax Media. If you are interested in the services of Hotwax I'll be happy to briefly answer questions and refer you to the Hotwax sales people.

Below is the rough schedule I have in mind and the general areas I plan to be in. I'll be changing it as needed to accommodate what people are available for. If you're around these areas and available around these times, please let me know!

17 April (Fri): Omaha NE, Des Moines IA, Kansas City MO
20 April (Mon): St. Louis MO
22 April (Wed): Indianapolis IN, Louisville KY (maybe Chicago IL)
24 April (Fri): Memphis TN, Nashville TN
27 April (Mon): Dallas TX, Oklahoma City OK

Please contact me directly by email if you're interested in a visit. If you know of someone else who might be interested in a visit, please make an introduction and if they are interested I'm happy to play along. My schedule is flexible on this trip so morning, lunch and evening visits along with normal business hours are fine.

There is no fee for this, but I won't refuse a free meal if offered. I'm traveling in a 40' motorhome, so hints for parking somewhat nearby are also appreciated.

I am planning to do this in other areas in the near future, but don't have firm plans yet. The next likely area will be the north-western USA in July (around the time and place of OSCON which I'll be speaking at on July 23rd in San Jose, CA).