This presentation is about how our UX team, which works on the desktop .com site for Best Buy, managed to create a process that supported an agile development methodology, while still keeping UX standards and processes in place. We know that this method may not work everywhere, but since we did find something that worked well for us, we wanted to share our experience. Please keep in mind while we're talking that what we've done won't work everywhere. But the point is that the Agile process and the UX process both need to make compromises to play well together. Maybe there are ways that this can happen where you work, too.
Within a year's span, we would plan, design, build, test, and launch new features and functionality, cramming them into an existing UI without the time or budget to look at a holistic experience. All design work was done up front, but we'd often have to de-scope a lot later in the year when the development process was going on, because usually we'd have asked for more than could get done in our timeline. Usually about half of our intended scope got delivered, resulting in a product we were less than proud of.Then we'd take the holiday freeze time to see how everything performed. Usually the problems we'd called out as risks would surface. But our project was over, funding was gone and the team that had worked on the original project had moved on to something else.
We were very contractor-heavy and that led to people leaving when the money ran out, or moving on to new projects immediately after launch.With this model we never really build a level of expertise within a subject matter (in our example, Cart and Checkout) to think deeply enough about what we really needed to do clean up or to optimize.
We knew this wasn't right. We were frustrated with the small amount of scope we managed to complete each year. We didn't like seeing people with all of that expertise just take off to go work on something else, leaving us to start over. We wanted a team of people who continually worked on our Cart and Checkout process. Who knew the ins and outs of the code, who understood the specific needs of the customers during this critical part of the purchase process, and who could eventually work toward a full redesign of this part of our site.
So we formed a new plan: An agile process, using two week sprints. In each sprint we would design, build, and test a piece of work that had been pre-determined by our product owner. It would be small enough to fit into a two week time period. We'd launch something once a month, that had value to the business and to the customer. It sounded like a great plan, so we got started.
Since we had been operating in a waterfall methodology for so long, and because nothing ever got totally done, or done correctly, we had a ton of debt: tech debt, UX debt, etc. We had business KPIs we needed to meet. We had requests coming in from outside teams to add new features and functionality. We needed to get organized. We spent about a month gathering every idea we could think of, or that others passed our way. We wrote our backlog, rewrote and reorganized it several times into Epics and stories. We attempted to size things, a few different times. We tried to write acceptance criteria.
We hired our teams. I was in charge of staffing the UX team. Typically on our projects we have specialists: an IA, a designer and a copywriter. These three roles work entirely collaboratively to produce our deliverables. We'd always had a lot of documentation to create for a variety of reasons. We had teams of developers overseas, we had to annotate for our content management system, and it helped keep really large teams in sync with what we needed for the UI. Plus, since we were mostly teams of contractors, we had to keep documentation in case anyone left, to help get new people up and running.While our intent was to stop doing all of that documentation and just sit as a co-located team, we quickly realized the documentation was still needed. Mostly because we were continually onboarding new people. And the stories our product manager wanted us to work on were much larger than a single sprint's worth of development.
Soon, we got really tired of this and just wanted to start building things. We thought that we could add the detail needed (well-written stories and acceptance criteria) sprint-by-sprint. It'd just be part of our kick off every week.
We soon found that it would take us a full two-week sprint just to do the research, planning and documentation for a two-week development sprint. This required us to work ahead. We also needed to sit with the developers while they built the code, to answer questions. This took a lot of our time. We also were responsible for doing our own UAT. This could take hours out of every day. We couldn't keep up.
And besides that, we didn't necessarily have a vision for what we were working towards. The stories we were assigned were mostly just small changes to existing UI. We were tasked with an eventual redesign. It'd be great if we could be making our incremental changes toward that larger picture. But we didn't know what that really looked like.
So, we got permission and budget to begin another UX track in parallel to the development track. This would offload our day-to-day working team and give us the vision of what we were working towards. This person would create innovative ideas for the big picture and work with our researchers to put them in front of our customers, learn from that process, and iterate. We hired another full-time IA to dedicate her time to this work. Adding more people to a team breaks some of the "rules" that people prescribe to the process. We're supposed to work in small teams. We're not supposed to have a lot of process. But that just wasn't going to work for us. There was too much to do, we had too many stakeholders (it's a big company!) and we couldn't work fast enough.
This is how I felt about Agile for a long time. We were not the monster. Other UX folks talked about trying to catch up and focus on a holistic picture
Alan Deutschman from Fast Company defines this concept as “…a …company where small groups can innovate and test their visions independently of everyone else. [Jeff Bezos from Amazon] came up with the notion of the "two-pizza team": If you can't feed a team with two pizzas, it's too large. That limits a task force to five to seven people, depending on their appetites. - http://www.fastcompany.com/50106/inside-mind-jeff-bezosCommerce started out this small, grew very large and then pared down again. One team is focused on the fundamental technology and back end code, we are focused on the front end.
Common opinion is that UX within Agile is hard. With the right team and good communication, it doesn’t have to be.Ours includes:An IA focused on innovation2 IA’s focused on the day to day work1 Copywriter to write and review content for all IA’s on the team1 Assistant creative director to work on innovation and ensure we are aligning with brand standards1 Designer for the day to day work and the AB tests we submit1 UX analyst for all – she tests the front end look and feel to ensure it’s functioning correctly and manages the CMS process
Pichler Consulting states: “A product roadmap can provide the following five benefits: First, it helps you communicate how you see the product develop.Second, it helps align the product and the company strategy. By anticipating how the product is likely to grow you can show how the product helps the organization reach its goals. This makes it easier to secure a budget for the next fiscal year.Third, a product roadmap helps manage the stakeholders and coordinate the development, marketing, and sales activities.Fourth, a product roadmap facilitates effective portfolio management, as it helps synchronize the development efforts of different products.Last but not least, using a roadmap supports and complements the product backlog.” The product roadmap is strategic, the backlog supports that work and is tactical - http://www.romanpichler.com/blog/product-planning/agile-product-roadmap/Last year we learned a few things about how to get things done. Don’t grab from a list of want to haves – make sure you have a prioritized list to work fromDon’t just start on a product request from another team – make sure it aligns with your roadmapDon’t work too far in advance to fill up the backlog because things changeOur process to determine the roadmap for this year:Created a big wishlist with the UX teamShared it with the product team and get their inputShared it with the technology team and get their inputPrioritized the list with further data:Call center data and customer survey data when applicableAssociated to analytics data and determined ROI – be conservative because it’s also the goal you’re measured onAdded industry trendsDetermine what you can feasibly produce in the next year and continue to show your team how you are all working toward that goal
Being given the opportunity to release new code every month doesn’t lessen the need for user research. Taking time up front to understand user needs gives greater confidence to the product once it’s released. There are three types of research I like to do. All of these types of research should include analytics data, any AB test results, call center data and customer survey data to determine what to test:One focused on a new piece of functionality One focused on improving existing functionality And a baseline test once yearly to make sure all the functionality we put on the site over the last year is working well together We do a lot of AB testing for a couple of reasons:To understand if our customers are interested in a piece of functionalityTo refine what we tested in usability with a larger sampleIn addition to the UX team, make sure to include the developers. We spend a lot of time problem solving in between sessions and the developer provides their knowledge of the code base, back end systems and their good ideas to the sessions
According the Wikipedia, user stories are defined as “…one or more sentences in the everyday or business language of the end user or user of a system that captures what a user does or needs to do as part of his or her job function…. User stories are a quick way of handling customer requirements without having to create formalized requirement documents and without performing administrative tasks related to maintaining them. The intention of the user story is to be able to respond faster and with less overhead to rapidly changing real-world requirements.” -http://en.wikipedia.org/wiki/User_storyWe struggle to write meaningful user stories that make sense to everyoneSome of our stories are transformed into epics when we dig into the detail We still document the whole kit and caboodle in wireframesWe have a schedule for reviewing stories in our group to slate work into releases:Kickoff meetingReview happy path design, determine if there are further scenarios to develop before putting in the backlogMake sure you have invited the right people to these meetings. For us it’s a mix of a developer, project manager, product owner, ux analyst, designer, writer and QA
Stand-ups are there to provide a daily status of what you’re working onThese are useful to the UX team about ½ of the time because we’re already sitting with and working closely with the team
Minimum viable product is the minimum amount of functionality needed to launch. We may have minimum functionality, but not minimum customer experience, so we wait until we’re sure it’s good enough for customers to see
Retrospectives offer an opportunity to take a look back at the work that was completed.We aren’t great at making these a formal process. There are people on our team who are great about seeing an issue, making a suggestion and improving the process.
Over the last year we have gone from being overwhelmed to working better togetherWe reworked the teamWedefined rolesWe created a roadmapWe informed the work with researchWe defined processWe used Agile when it made sense for us
We will continue to iterate on the process:Better codeBetter contentTweakand reconfigure the teamA different way of working togetherReconfigure the team slightlyWeekly releases
Make long term and short term goalsGet a developer buddyDefine your role to the team and ask them to define theirsIncorporate user researchRevise goals as needed
