If you were developing a new application where would you start? Would you first create all the supporting frameworks for security, logging, etc.? Would you create the login screen?
Most of the time when I've worked with new Scrum teams, they initially order the Product Backlogs sequentially along with the flow of the application: Frameworks, Login, User Administration, etc. This seems like a natural flow to creating components since it normally maps to the natural flow of the application itself to some extent.
But one of the basic principles of a Product Backlog is to prioritize by value. What do we mean by value? That could be many things such as ROI, risk, regulatory compliance, etc.
So, is the logging framework a highly valuable backlog item? What's the ROI? Little to none unless your company's primary business is selling logging frameworks. What about risk? Do we mean that it is risky to NOT implement a logging framework? No, when we push items to the top based on risk it is because we are not sure of the technology we are using or the approach we are taking so we want to implement the item sooner so that we can prove the technology or concept early on in case we have to change. So is a logging framework risky? Hopefully not. If you are a developer, with any kind of experience, you have implemented logging countless times and probably know a few out of the box frameworks you can easily integrate. So, it looks like we are simply building the logging framework first because it is our idea of a natural flow of building software applications.
But if we build all of the support frameworks, and the login page, and user management, etc. first, then when do we get to the real functionality that we are actually selling to our customers? Probably late in the game and uncomfortably close to our deliver date. Which means if we have any issues with these features we have less time to react to them and more likelihood we will have to work nights and weekends to "get er done" and probably have some quality issues once we do deliver.
So then what should we build first?
Ask yourself- 'what is the main feature set your customers are going to be paying for?' Build that first.
"What? You're crazy! I can't build that feature until I have all that other stuff you mentioned before."
First, I am not crazy, my mother had me tested.
Second, you can build those types of features first. You will have to unlearn a good bit of what you were taught as a tried and true architect in the layered application design. I am not saying you will not eventually build those things because obviously most of it is something you need to deliver a customer ready application.
I had a client in the financial services industry who was building a new product. Their initial user story sessions started exactly as I described above where the group started listing off the features they needed to build in the order of the application flow with several framework stories at the very top. I asked them what was the primary feature their customers were paying for in the application and they said transferring money from X to Y institution. I pointed out that this would be the best candidate for one of their first stories to tackle and after some convincing they caught on and ran with it.
Once we started breaking that down into smaller stories (since the original one was identified as an Epic) they once again start prioritizing the components of the feature sequentially: create a new X institution, list existing X institutions, create a new Y institution, etc. The story for the actual transfer of funds from X to Y was fairly far down the list. I once again asked what the most valuable part of this feature was to their customers and it was unanimous for the funds transfer.
Then the common questions rose up like: "How can we build the transfer from X to Y when we do not have the functionality to manage X and Y?" Do you sell X and Y management solutions to your customers? "No." You can create the simplest solution to populate X and Y even if it is just a database script. This way you can concentrate of getting the actual transfer part correct and demo that to your stakeholders and customer representatives first. That way you get feedback right off the bat and expose any implementation issues early. Then you can start to get to the other items such as the ability to add X and Y, login, logging, etc.
"You're crazy! How can we design the application architecture without knowing how we are going to implement security, logging, etc.?"
First, I am not crazy, the voices in my head said so.
Second, when building applications incrementally in an Agile environment your should follow the practice of emergent design where you build applications with just enough framework to support the current functionality but using an approach that allows for extension or replacement of these frameworks in the future. This approach includes the principles commonly called SOLID that help build decoupled, independent systems. Understand that there may be some level or rework, but just as we want to embrace change in the requirements in an Agile environment, you have to accept a certain amount of change in your architectural approach as well. We want to build features first and just enough framework to support them now and in the foreseeable future. Too many architects (me included) have gone all "mad scientist" when building glorious architectures that solve all the world's problems (even if they did not ask us to).
To further illustrate my point we talked to another team that had implemented Scrum on a previous project, but had built their features in that logical sequence. They were running on a 4 sprint release cycle and when they spent the first 3 sprints building frameworks and supporting functionality, they found that they had issues when trying to implement the core user functionality. By then it was too late because they were in their last sprint before their release. So what happened? They missed that release date and lost half the team in the next sprint to finishing the functionality. On top of that the rush to get it done left them with a ton of production bugs.
Now all of this is easy to say and harder to execute. Slicing up features this way takes practice since it often goes against your historical methods. While emergent design and SOLID principles sound great for building these types of systems, they do demand a certain level of development expertise and experience. But it is achievable and allows you concentrate on delivering your most valuable items first without weighing your application down with over bloated design from the start.
So when you start to prioritize your Product Backlog, remember to concentrate on value and not fall back on sequential flow.