Posts Tagged ‘methodology’

Axosoft Customer Survey Results

July 21, 2008 9 comments

Last week, we sent out a carefully crafted survey to our customer list hoping that we’d get about 40 or 50 responses to help us better understand what we’re doing right and what areas we could improve.

To our surprise, we received an overwhelming response from over 360 customers who completed a fairly lengthy, 20+ question survey. The results were absolutely awesome — surprising in many areas. They highlight a number of things where Axosoft can clearly do a better job and they also point to a number of things we are doing right. I thought it would be interesting to share some of the numbers visually, so I threw them into Keynote, charting and charted and graphed the responses.

Here are the results:

Q. Rate Axosoft in the Following Areas:

These results were very reassuring. My raw immediate reactions were as follows:

  • Holy cow, we have way too many “Never Used” responses. We need to do a much better job of getting users to know about the existence of the Axosoft Community siteVideo Tutorial Podcasts and OnTime Overview Videos, all of which are excellent resources and free!
  • Although I never want to see a survey result of “Unhelpful” or “Terrible”, it was good to see that we had fewer than 6 such responses total, combined for any question. Not bad when you consider more than 360 people responded. Now that we have a baseline, we’ll have a goal to reduce even further terrible and unhelpful responses.
  • It was great to see the Axosoft Sales Reps, the Axosoft Web site and the overview videos get such awesome responses from customers who used those services.

Q. Which version of OnTime do you use:

We were pleasantly surprised with the results of this question as we did not expect nearly 80% of our survey respondents to be using OnTime 2008. This told us we’ve done a couple of things right: a) OnTime 2008 was a compelling upgrade for the vast majority of users and b) We’ve done a better job of making upgrades easier than previous versions.

Q. Which OnTime Products Do You Use?

Here we got confirmation that OnTime Windows is by far the most popular client type we offer. The high use of the OnTime Customer Portal (41%) was a bit of a surprise as was the 13% usage of the OnTime SDK. If 13% of our customers use the SDK, it deserves to get a bump in priority for continued enhancements.

Q. Which OnTime Functionality Do You Use?

The findings to this question were in-line with our expectations. It was great to see adoption of OnTime as a HelpDesk tool has already reached 50% of survey respondents. HelpDesk functionality was introduced in V7, so having 50% of our customers using OnTime as a helpdesk solution was great to see. Similarly, Wiki usage is growing rapidly. Having just introduced Wiki functionality in V8 only a few months ago, it’s great to see more than 20% of our survey respondents using the team Wiki.

Q. What are Your Favorite OnTime Features?

OnTime’s ability to automate workflow and to enforce team processes was listed by the most people as one of their favorite features. We’ve spent a considerable amount of time making sure the workflow functionality in OnTime is extremely flexible and it was good to see that its being used and loved by such a large group of people.

To balance out the favorite features question, we also asked which features users felt were missing or needed more work. The information there was extremely interesting, but publishing those results publicly would be handing a little too much ammo to our competitors (sorry guys).

Q. Which Smart Phone Do You Use?

The results here were predictable. But far more interesting was which smart phone users planned to purchase:

In my opinion, this puts the iPhone debate to rest. More people plan to purchase iPhones (even this early in the game) than all other smart phones combined. iPhone has already won the mobile platform wars.

Q. Which Browser Version Do You Use?

Two things are very interesting here: 1) FireFox’s adoption rate is growing unbelievably fast with nearly 34% of our survey respondents using FireFox and 2) FireFox does a much better job than Microsoft in getting new version adoption as FF V3 has just barely been out for a few weeks and already more than half of the FF users are on the new version.

Q. Which Version of SQL Server Do You Use?

It was good to confirm that the vast majority of customers are on SQL Server 2005. However, due to a still large number using SQL 2000 (~24%), we need to continue backwards compatibility for these users.

Q. Do You Have Any Mac OS X OnTime Users?

Considering Axosoft’s focus has traditionally been on Microsoft development teams, having a 12% response that yes, there are OS X users of OnTime was a bit surprising to some, but confirms the trend that users are migrating away from Windows at alarming rates. Note: the exact wording of this question was not about Apple hardware, but specifically Mac OS X.

Q. How Do Your Mac OS X Users Run OnTime?

As suspected, the majority use browsers. But still, the number of users utilizing a local Windows virtual machine was interesting.

Q. What Development Methodology Do You Use?

This question confirmed that the majority of software development teams don’t follow a development methodology. But it also shows that of those who do, Scrum is one of the top methodologies getting adopted by teams. At Axosoft, we’ve been very intrigued with Scrum and the philosophy behind it. Scrum is very much inline with our own development philosophy, and we hope to improve OnTime even more so it addresses the needs of Scrum teams even better in the future.

Q. Would You Be Interested in an OnTime User Conference in Scottsdale, AZ?

We weren’t sure what to expect with this question. But it was nice to see that nearly 1/3 of survey respondents would be interested in an OnTime User Conference. Looks like our Marketing and Training Team have their work cut out for them. Stay tuned.

Q. Does OnTime Truly Help You Ship Software OnTime?

At first, I was a bit disappointed with the results to this question. Sure, OnTime is helping about 6 out of 7 teams that use it ship their software on-time, but what about the other 1 out of 7? Why are they even using OnTime, if it doesn’t help them ship software on-time?

Fortunately, we had a follow-up open ended question that asked users why OnTime isn’t helping them ship software on-time. The results were more reassuring. Some of the respondents indicated that they were relatively new to OnTime and hadn’t yet shipped a project since they started using OnTime. Some felt that OnTime did a great job, but their project completion issues were unrelated to OnTime (management, executives, customers – you know the drill). A small fraction of the 16% for whom OnTime was not yet hitting the mark gave the indication that OnTime is missing some important project visibility information that they find important to shipping software on-time.

Improved project visibility will be a major focus of the next major release of OnTime.

Countries of Origin:

We had users from more than 30 countries fill out the survey. With about 44% of survey respondents from abroad, this was an exceptionally balanced survey that represented a good sliver of our customer base.

Even More Information to Analyze

We have a lot more information to analyze. Several of our questions were open-ended questions where users could write their thoughts about Axosoft, OnTime and other related items. The responses to those questions were and will continue to be extremely useful to us as we plan for future versions of OnTime.

I wanted to also thank all of our customers and especially those who took the time to fill out the survey. The feedback we have received is instrumental to the continued success of Axosoft as a company and the  OnTime product line.

Now, let me know if this poll was helpful:


Building a Great Team

November 12, 2007 6 comments

A Good Team is Critical to Success of a ProjectThe centerpiece of any successful development project is the team that builds it. There is no other single most important contributing factor to building great products. No tools, no development methods, no amount of money and no amount of time can substitute for the importance of an exceptional team if you plan to create an exceptional product.

Some in our industry operate under the assumption “with a good system and a fine-tuned set of processes, you can build anything with a group of average Joes.” If that sounds like your company, congratulations! You must be that average Joe. The assumption that with the right systems and tools, anybody could build great products couldn’t be any further from the truth.

Only driven, motivated and talented teams can make great products. Heavy systematic methodologies of product development and processes can only slow down those teams to ensure that the slowest person on the team can keep up. Process-heavy systems are designed for the weakest team member and therefore must slow down the pace of all other team members to ensure that few mistakes are made by those who are not exceptionally skilled. The cost of such systems is incredibly high.

See if this sounds familiar: Your team is fast at work on the latest project. Due to the size and importance of the project to the company’s success, the executive team is providing the budget necessary to expand the team. It’s estimated that your team will need to increase the “headcount” by 5 or 10, in order to meet its project deadlines. To quickly ramp up for the new required headcount, the bar for qualified candidates falls nearly as fast as the team’s cohesiveness. Within a short time your team successfully adds 10 new “heads” to meet the aggressive project schedule.

To ramp up the new members as quickly as possible, they have been given the proper one-day overview of the project. They weren’t even asked to write code until Day Two. Soon these new team members start contributing thousands of lines of code. Productivity seems at an all-time high. Tasks are getting done in record time and the team now seems to be functioning on all cylinders. If you’re lucky, within a couple of weeks, the first set of problems start to reveal their ugly heads. All kinds of new nasty bugs show up in QA (assuming, of course, that there is a QA team). Senior team members have to take time off of their tasks to track down these nasty issues. After days of searching and going through weeks of code line-by-line, some suspected line code are identified. The first question asked is “who wrote this crap?” and so the process expands.

In order to find out who’s responsible for poorly written code in the future, “checkin” rules requiring detailed comments are put into place.  Additionally, every single line of code is now required to go through a complete and formal code-review to help reduce errors.

The team’s pace changes drastically. Things slow down and soon it’s obvious that the deadline targets of the project schedule are impossible to meet. At current pace, you easily need to increase the headcount by another 5 to 10 to make the ship dates.  But, even that is unlikely as a frustrated, but competent teammember finally speaks up and reminds the executives (and everybody else in the room) that 9 women can’t make a baby in 1 month.

Finally, everybody comes to accept the missed deadlines and the pushed back release dates. Life goes on…slowly.

There is a Better Way

There is, of course, a better way. Among engineers and project managers, it’s common knowledge that a really good engineer can be 3, 5 or even as much as 10 times more productive than a bad engineer. What a load of crap. You’re probably thinking I’m nuts. You might even be thinking that you personally know software engineers who are 3 or 5 times more productive than the guy in the next cube. What you don’t realize is that you are probably discounting their productivity.

It’s possible for a good engineer to be infinitely more productive than the engineer sitting next to them. How is that possible? Consider this: in 1975, how many engineers could have single-handedly done what Steve Wozniak did in designing the Apple I and later the Apple II? What about the search algorithms used by Larry and Sergey (Google)? I know that if you had given me 10 years, it’s not likely that I would have thought of page ranking or search in the way that Larry and Sergey did. So how much more productive were they? I’d say about ($)150 billion times!

The problem with comparing people in terms of multiples of productivity (such as person X is 5 times more productive than person Y) is that there is an underlying assumption that both person X and person Y can do the same job. That is a huge and usually wrong assumption. Two people might be able to do the same job and if one of them is generally 5 times more productive than the other, that probably means there is a whole realm of things that is possible for one of them that’s impossible for the other.

With that in mind, if you were working on a hardware project and estimated that you needed 10 more engineers to get the job done, would you take 10 “Average Joes” or would you settle for 1 Steve Wozniak? If you were working on a Search-related project, would you take 10 “Average Joes” or 1 Larry Page or Sergey Brin? If the answers to those questions seem obvious to you, so should the importance of building a great team. It’s not about “headcounts.” It’s about talent. So to create a winning team, keep a few simple rules in mind:

  1. Hire the most talented people you can possibly find. Don’t hire someone if they don’t meet the bar you’ve set. I know it’s extremely tempting to lower the bar when it’s hard to find good people. Remember that a bad hire could actually have negative productivity!
  2. Create a system that’s flexible and light in process. Implementing any kind of canned methodology that has steps outlined from A to Z on how to create products and manage teams ensures that your team will operate at a low efficiency. A more dynamic and light-weight, low-process system is ideal when your team is already talented enough to know how to do the right things.
  3. Eliminate team members that consistently fall below the bar. Don’t kill them. Just fire them or move them to a different position if you can. If you can’t get rid of team members due to corporate culture or red tape, give them tasks that are non-critical to the success of your project. Often times you can increase productivity faster by removing team members than you can by adding them. If you find that you’re adding process to your efficient system because of 1 or 2 people who consistently need detailed guidelines to get work done, those are the guys I’m talking about.
  4. Create a culture that appreciates talent. Talented people like to work with other talented people. That’s how they’re satisfied. By giving them freedom to do the right thing and taking obstacles (and dumb people) out of their way, you’re creating a culture that appreciates and rewards talent.

By putting the emphasis on the team rather than the process, you’ll create much better products at the end.

Why Software Methodologies Don’t Save Software Projects

November 9, 2007 7 comments

Rigid Systems Can’t Save Software ProjectsAccording to Wikipedia, in Software Engineering, a Methodology is defined as:

A codified set of practices (sometimes accompanied by training materials, formal educational programs, worksheets, and diagramming tools) that may be repeatably carried out to produce software.

Unfortunately, that definition is believed by many, and it’s dead wrong. If you believe it, your projects may be doomed to failure.

Methodologies are systems for the creation of things. A system designed to manufacture cars at Ford or hamburgers at McDonalds is a methodology (although not a software methodology). Ford manufacturing plants produce new cars through an assembly line where each phase of construction is extremely well defined. It’s defined to the point that even robots participate in various phases of construction. You can interchange people who work in the assembly line with relative ease and almost no impact on the end result. There’s quality assurance at every phase to ensure that each benchmark is within the predefined limits. All of this results in the manufacturing of nearly identical cars, in every single instance. It works great.

Likewise, McDonalds has a similar strategy for making hamburgers. The steps to create a burger are well defined. If you change the cook, the burger will still be the same with the same amount of materials, same ingredients, same cooking temperatures, same sauces and the same wrapping. McDonald’s burger-creation methodology results in the customer enjoying the same burger, every single time. Their success is proof that the system works and it works well.

Methodologies Don’t Produce Software

So why not a system for the creation of software? If Ford can do it for cars and McDonalds can do it for hamburgers, why can’t we have a system to produce software? The answer is that we can. The catch, however, is that the system can only re-create software that has already been created. The system cannot produce new software just as Ford’s methodology for creating new cars will never create a different kind of car. It’s designed to re-create the exact same gas-guzzling, poor quality, tire-exploding and boring car in the exact same way that you have come to expect from Ford. So it is at McDonald’s. The system doesn’t create a new kind of hamburger. It simply re-creates the same bad-tasting, low-quality, high-calorie hamburger that you’ve now come to expect from McDonald’s.

In the same light, a software methodology cannot produce new software. This concept is a major surprise to a lot of companies and the US Government, who after spending hundreds of millions of dollars on software projects that end up being canceled — projects that should have been fool-proof in their success. After all, they used the most concrete software methodologies known to man-kind (or to Grady Booch). How could such software projects fail?

Learning that methodologies or well-defined systems and processes can’t create things is important. People create things. Systems and methodologies can only re-create things, but only after people create them first. So what about successful software projects that have employed rigid systems and software methodologies? Easy. Such projects have succeeded in spite of the methodologies used, not because of them and it’s virtually guaranteed that a less restricted team would have accomplished the same task in less time.

New Trends in Software Development

The world is slowly coming to the realization that people are the most important part of any successful new software project. No systems or processes can substitute for your top software engineers. Take your own software projects as an example. Would you give up the top one or two team members or would you give up your software methodology? My guess is that the answer is a no-brainer. Even in a team of 100, losing the top couple of team members would be far more detrimental to the success of the project.

You have undoubtedly heard of new software methodologies such as Agile and Scrum. The popularity of these new light-weight systems is exactly because of the reasons I have explained above. They put the emphasis on people. Is it any surprise that trimming down detailed, process-heavy, software methodologies has produced much better results? Of course not. These new systems are putting the emphasis on individual team members, finally coming to the acceptance that the individuals are far more important in the creation of new software. The systems should only support the individuals, not get in their way. Unfortunately, even these new methodologies are slowly becoming heavy. Scrum is becoming better defined by the day and the more it gets defined, the more Scrum Masters who insist on having a burn-down chart from day 1 on a new project, the more such projects will suffer.

Finding the Right Balance

Like everything in life, there is a balance that each team must obtain between systems and processes and team and individual freedom. Each project is different. There is no one system that can meet the needs of all teams and no single implementation of a system that should be used instead of another. Google’s founders didn’t start with a methodology. Neither did YouTube or FaceBook. Even well established companies like Microsoft and Apple didn’t (and still don’t) have rigid systems in place and they provide significant team freedom to choose the right amount of process for their projects.

So unless you have a lot of free tax-money to waste or if you work on human-safety systems, reconsider the weight of your software methodology. In software, lighter is usually better.

%d bloggers like this: