I've been struggling over the last couple of weeks trying to define a good measure that would help to show if a project was successful or not. As you know, I'm a founder of FiveMinutes and ShoutEm and both companies are almost exclusively working on projects related to mobile apps. It is important for us to have an exact measure that will give us an answer to this question.
You can't manage what you don't measure.
So the old saying goes. Of course there are exceptions to this but I tend to think that most things are measurable. In the past couple of years I've started to measure many things at ShoutEm and while there is always tons of room for improvement, we get so much back in return in terms of product quality, employee motivation and being proactive in improving key metrics, that the numbers work and, most importantly, they are focused on the right things.
Why is measuring so important, even when it looks like there is no single thing to measure or that some things are too complex to be easily reduced to one simple number?
Have you ever argued about something? Usually people do not agree because they have different opinions. For example, one of our employees will say that our mobile app project wasn't successful because it wasn’t downloaded much. Another will say that it was a very successful app because it works very fast, its design was revolutionary, but the users failed to understand it, etc. You will never end that argument because both are right and wrong at the same time.
There are no Gods!
I really want to build a culture where people’s opinions are completely irrelevant. Every single person thinks that their thinking is right. Every assumption should be tested and proven using data. There are no Gods, except me :). I admit, it is not easy: I also struggle with ending an argument when not everyone accepts what I have said. However, I can see the benefits of encouraging people to come up with new ideas and to experiment. I'm all for it. But in the end, each and every one of those ideas has to be tested and valued somehow.
So let's get back to project development in our type of business. Do you think that if we had an exact measure that we all agreed on before we started a project, it would help? I know for sure it would. When in doubt, just check that number on the screen. Does it say 3/10 or 9/10? There is no mistaking whether it is successful or not, but more importantly we know exactly what we need to do to fix it.
So what is a good measure of an app’s success?
This is what I came up with:
Client targets [0-10]: where 0 is 0% and 10 is 100% of target achieved. Targets usually are revenue, downloads and user engagement.
Average app store rating [1-5]
Client team’s satisfaction with the project [1-5]: Average rating from members of the client’s team with overall satisfaction of the product delivered, collaboration, timing…
FiveMinutes team’s satisfaction with the project: [1-5]: Average rating from internal members of the client’s team. Satisfaction with the results, work load, collaboration with the client…
With those numbers, I use a formula to calculate a single number. It weights the average app store rating and user engagement a little bit more and client satisfaction and team satisfaction a little bit less. This is because I think end user opinion is the most important and that the client will be happier if there are good results rather than we just did what was asked for. The team might suffer from time to time, while that is not great, for the company it is more important to deliver the highest quality projects continually over time.
So here it is:
App Quality = (Client Targets + App Rating + Client Satisfaction + Team Satisfaction) / 4
Anything below 4 would be considered unsatisfactory and would need improvement. Additionally, the formula can be modified if the project is very different to a typical project.
How do you measure the success of your projects? Let me know in the comments.