This post is inspired by Abdel’s comment here in which he asks for some specifics about how we actually apply Scrum in a BI/DW environment. I’ll answer the questions directly and add a few bits and pieces as they come to me. Of course this is just one perspective and as with most things in life, we’re learning as we go but I am more convinced than ever that Agile is the way to go for BI/DW development.

How do you specify teams?

We have little option here. There are 3 of us in our BI/DW team so we are always involved. We’re doing a lot of work with the Planning department at the moment so they are also in the team most of the time. There are of course times when we need to pull in a boutique developer – someone who has a special skill or expertise we need for a specific purpose – so they become part of the team for perhaps just one or two sprints.

The biggest hurdle we’ve had to overcome is identifying and getting a product owner, someone who not only has the vision but can also make decisions. These people are in high demand and are few and far between in most organisations. If they do their job properly they also need to do a lot of running around and consulting in order to make sure what we’re working on is indeed what we should be working on. We’ve found one now, I just hope we can hang onto them.

How do you choose the sprint process or sub-process?

This is where the product owner comes in although generally in practice we just know what it is we are supposed to be working on. We still have some kind of governance structure in place and I imagine you do too. There will be someone saying ‘you need to get x, y and z delivered’. On this basis you can say we’ll work on x for 2 months and then review what has been achieved. In that 2 months you might do 7 sprints and each one will chip away at the backlog of requirements until either your product owner says they have enough or you have some other higher priority to focus on.

When is it ok to stop your datamart development and move on?

I definitely think it is better and easier to focus on a delivery area for a period of time rather than saying you will deliver a product in its entirety – you’re getting into dangerous territory if you do the latter as you can’t accurately estimate when that will be and as we know, the scope and priority of the product delivery components will change constantly from day one. You specifically asked about delivering a datamart – you might for example, choose to deliver a single fact table with maybe 6 key dimensions. You might have other stories on your backlog that you might not get to. These could be things like:

• deliver an aggregated fact table or some kind of high-performance summary table
• migrate historical data into the core fact table
• add new attributes to some of the core dimensions
• add new dimensions to the model

This is where priorities and time come into your equation. If you can, you may keep delivering all of the above, or you may stop and put your energies into something entirely different. The approach manages whatever is needed for your organisation. The main thing is to ensure you deliver something of value in each sprint. Dimensional models are by design very extensible so this again lends itself very well to the above.

Sprint Duration

We’ve moved to 10 business day sprints, that works much better for us. A month is just too long and we run out of steam after the second week. With a demo day, we’re now not starting sprints on the same day of the week each time but that seems to be working out ok. Keep the flexibility to try new things, I think that is the most important point here.

Daily Stand-Up Location

In an attempt to improve collaboration we are having our daily stand-ups in the business area that we are doing the development for. Although this is a little bit inconvenient for us, it seems to help, we are working with them on their terms and often it is the best way for them to see and hear what we are doing. It also means they can easily get involved in our daily meetings and I think sends the right signals.

• Previous Entry: Tableau 5.0 available now
• Next Entry: University iPhone Stories