Pilot Groups or Sandboxes: Which Is Better for Your Implementation
It’s always a good idea when we’re creating change or implementing something new to make every attempt to mitigate our mistakes. In software development, many organizations use a process called agile to do that. Agile is when a larger process is broken down into smaller steps or milestones. After each step, the development team debriefs their progress and makes any necessary revisions moving forward.
The idea behind agile is, if we evaluate our work at milestones, then we might mitigate a future mistake. But today’s post isn’t about agile. It’s about implementation and evaluation.
In the learning and development function, instructional designers build in a testing phase to make sure a training session flows the way it was designed. That testing phase is often referred to as a pilot group. The pilot group participants not only attend the training, but they provide design feedback. This allows the instructional design team to make final adjustments before the program goes live to the entire organization.
During last year’s HR Technology Conference, I heard a new term – sandbox environment. My take is that sandboxes are another way to test before going live. Now, I will admit that everything I’ve seen to date on sandboxes is related to software development, but I do wonder if it would be something that we can use outside of it, given that we talk of agile quite successfully outside of software development.
Just in case you’re wondering, the term sandbox comes from the children’s play area. A sandbox simulates the beach or some other locale. Children are able to play and discover – safely – in a smaller version. Software sandboxing does the same thing. It creates a smaller environment that can be used to test the software before it goes live. It does seem to me that pilot groups and sandboxes – while they have the same ultimate goal – are two different things.
Pilot groups present a finished product to a group for feedback. The feedback group is typically composed of part of the audience that the product or program has been designed for. The program may or may not have changes made based on the feedback received.
Sandboxes are traditionally used by the development team. The product isn’t necessarily ready to go live during the sandbox phase. But like the pilot, it does create an opportunity to tweak the product before its finalized.
One thing is certain, whether the organization decides on using pilot groups of sandboxes, they will want to establish the following:
- Agree on a common testing protocol. Organizations will get better at testing if they use a consistent process. Testing involves work and the organization doesn’t want to retrain people every time.
- Have a testing goal. What type of information is the development team looking for? How will that information help with the overall implementation plan? Everyone needs to agree with this.
- Create a baseline for feedback. What I mean here is tell the testing group what the development team wants and how they should send their feedback. Good feedback will help deliver a better outcome.
I can see organizations trying to use a sandbox environment both in place of and in addition to a pilot group. That could be a good thing regardless of the type of implementation the organization is experiencing. These types of testing protocols could be valuable for software development, training programs, or a new organizational initiative. The question becomes which one is the best for your particular implementations.
Image captured by Sharlyn Lauby while exploring the streets of Fort Lauderdale, FL
12