What to measure when one has agile/ scrum based product development? Many we have met, measure burn down charts, but not really that much else. What should you?

"90% of the startups fail" a recent Fortune review tells us. The biggest reason for this, based on the same study, is that startups create products or services that nobody wants.

It often starts that the team has this great product idea, spends rather long time to make the features and all, and then tries to put it out "to be found" by the customers. But the channel to sell and promote seems to be the problem. Some interest arises but not substantially enough to raise the user base for the product or the service. One tries to improve the situation by changing the product by adding more helpful features into it, but no real big change seems to take place. "If customers just would be sensible and see how good this product is".

Our Agile operation might not differ so much, in fact. We have learnt from the agile books and seminars that the way to do it, is to have a backlog driven tight agile/scrum based product creation where the order of the items are the order of importance that we believe customers want their functionality. And of course if you are making a product to a specific customer order, you can validate this with them by asking.

But in many cases customers are nowhere to be seen, and if one searches truly new things, the customers do not know what they want. Horseback riders did not know they needed the T-Ford before it was there. So therefore the development teams need to follow their intuition on what is important and which kind of a product to create. See the analogy with the failing startups?

Lean startup methodology was developed originally for the startups few years back, but recently it has been applied into many other cases as well. Companies like GE use it to improve their innovation and radical product concept capability (you can google FastWorks).

But it can and in the opinion of the writer, should be used in agile. The logic goes so that companies should more and more find innovative new things. New things that create value are such that customers are willing to pay a premium, and no competition exists. So why would you want to market follower and do me-too-products with low margins?

We will examine in the following blog posts how you can easily improve process capability and create innovative new products just by enhancing it with lean startup methods.

 Some keywords will be: MVP, Vanity metrics and Pivot.


In the last post we examined how to use Lean Startup practices. We assumed a mailbox based banking product, and it looked not to be enough of an innovation to succeed. And that Google's Pony Express has somewhat a similar idea. (It is not public so we do not know exactly).

Additionally we claimed that you might have the wrong approach, if your front-end solely consist of backlogs owned by PO (Product Owner in Agile methodology). So even if you fulfill agile methodologies by using a PO and backlog, you might have missed it.

Let's go forward in our thinking and examine these further.

Our textbooks have taught us that the road to success is to fulfill customers either known (or at the time un-known, latent) needs with our product. But who is my customer?

Lack of customer focus is offered as a reason for Nokia (mobile) downfall in many articles written on the topic. But one could argue that Nokia did focus on customers. And it did. It had hundreds if not few thousands of operator requirements fullfilled in Symbian SW. Remember at that time the phone business was such that in order to get into subsidized operator sales channel the operator technical team was the gatekeeper. You fulfill our reqs, and you get into the sales channel.

Right, but the requirements were the operator technical team's small changes to Nokia's requirement set, not the holistic user experience that the end user wants. What Apple did was not to mind any of those and they by-passed the operator channel completely. Because the opinion of the operators was not seen important. Looking the results one might argue that Apple selected customer that was "more right".

So how should I approach this?

Let's remember here the Geoffrey Moore's technology adaptation life-cycle (you can google it) defines the user segments as: 

  • Techies - just try it
  • Visionaries - the lead users
  • Pragmatics - follow the herd
  • Conservatives - use only proven things
  • Skeptics - no I do not believe in it

You can keep these as your default user classes. You and your PO should use them as to understand your customer and what would be your targets. Now often the technology apps and innovations are directed to techies, but be aware that if you have difficulties in expanding your user base, you might target the wrong segment and needs.

Let's come back to this email based banking example. It is perfect for Google, they like owning your mailbox. It adds stickiness into their email and extends it to new business areas. But you should think it other way around. How to find a banking solution for the visionaries and pragmatics, which you want to target as they are a large potential user base. Everyone in that segment has a smartphone, or at least a camera in the phone. So you should test a banking app where you take few pictures on the bill you should pay, and the app has magic image processing technology which combines the pictures in a one clear shot, and sends that into the bank to be paid. Could be useful?

Is that a killer app? We do not know. Make a Lean Startup MVP (Minimum Viable Product) to test your idea. MVP is to be used to see if your idea flies. The MVP does not need to work, but it is more for validating our grand idea here above. And if some aspect needs to be changed, you can test it again in a cheap and quick way.

Once you know you customer and have validated you visions with them you are in the clear and can start the more traditional product development cycle.

It is not always obvious - Let's be careful out there!


Most of the readers are familiar with Agile methods, and most companies even know how to implement them in practice. (Not all do, but that is another topic and another post). Agile methodologies have improved SW development efficiency as an operation, one can argue. But how to be sure that what one develops delivers the best VALUE? Or that one creates truly innovative products, not just copying features from someone else? In short: how to succeed?

I told in my last post that even if one has a proper agile operation with themes, Epics well basically PO (product owner) and prioritized backlog, one can miss the mark. WHY? You might have the wrong backlog or the wrong product.

Lean startup methodology has a few but important, interesting concepts to add in to your product development front-end. Let's examine them further:

Of course, one should always to have a vision to start with. You have an innovation or an interesting concept you think will be big. You might be right, but how can you tell? Ask the user? But what and how? We do not have any customers yet.

In lean startup methodology one creates a set of assumptions that one tests with a MVP - minimum viable product.

Lets take an example. I have this vision that people could pay their bills straight from their email. The insurance/ gas etc company emails the bills, and they get paid digitally by users. First I create a set of assumptions that I need to test:

1. Banks would like this as less people would visit their physical branch offices and payments would happen remotely. Less money spent on bank walls.

2. People would like this as they would not need to visit the banks in person. Better experience as one needs to queue for a long time to get the turn at the bank desk.

3. Both banks and people would be willing to pay for this.

Now I need to test my assumptions, as we know not all we dream about are actually true. I need to build a MVP or several to test out my assumptions. You might spot that all the assumptions are on a quite high level, and a likely answer to those might be "yes, but it depends". The real product innovation lies in the implementation, and a more concrete test MVP is needed.

Let's assume I build a MVP, a product which is an email extension that fires up by clicking the bill attachment, and then guides me through the login process to the bank etc and lets me pay the bill. 

OK, now I need to get feedback to the concept and assumptions by interviewing people. As this is a fictive example, I do not have the real data.

But I would argue that answers would be:

Yes, this is fine, and the Bank likes this concept, but the innovation here is only that the attachment opens the bank's web service. No more. One can open the bank's current web bank page even without and pay the bill.

Not enough willingness to pay for the service. Users would think that banks would need to offer this kind of service for free or as a part of a service package. No space for startup revenue here.

As a total summary of this innovation it is unlikely that the assumptions would be met. People with a laptop and an email account, is not probably that segment of people that visits the bank offices and builds the long queues by wanting to pay for their bills in person. So even with an innovative implementation the end benefit would not have been fulfilled. So people with email account can use the bank's web service.

NOTE that the strategy of asking the users which features to add, we would be after the wrong segment. (We look this in more detail in the next post). 

So it is time to PIVOT. Pivoting is a lean startup term, and means that one needs to change the course - meaning to change one or several assumptions. 

In the next post we figure what to do next. This spec did not work for us.

- Jukka

ps. Google is working on this kind of banking service - you might want to google Pony Express

ps 2. Note the question 3. Not many innovations and startups have a solid concept for monetization. You should build that in to the set of assumptions. You test, you know.