“My elearning firm was awesome … until they weren’t.”
Every homeowner loves their contractor when they first hire them. It’s not until the project gets underway and something goes sideways that they realize they made a poor decision. The same dynamic exists with learning agencies. With a great sales and marketing team, it’s easy to fall in love with a learning agency. But once the hire is made and the sales team moves on, what you’re left with will determine whether the relationship is a success.
Mistakes buyers make
Most buyers that I’ve seen take the same basic approach. They first examine the agency’s portfolio and case studies. They try to learn more about how the agency thinks about and generally approaches learning. They may learn about the process the agency uses. These are all very important things to evaluate in the vetting process, particularly when you’re just trying to sort through initial candidates. But when this is all we do, as is often the case, it leaves us open to some real problems.
1. Over value visual appeal
Reviewing past work is a critical early step. If you don’t like what you see, you should move along. But seeing things you like can be highly misleading. Because most agencies are not allowed to publicly display most of their past work, they usually have to show screen grabs with verbal descriptions. This is helpful, but it prevents you from really evaluating the true learning experience. Worse, it encourages a bias toward overvaluing the visual appeal of the work. This is analogous to being blinded by the charisma of a job candidate who lacks the skill needed to do the job. Visuals are important, don’t get me wrong. But the world is filled with beautiful courses that nobody learned from. It’s entirely possible to create high impact courses with little visual appeal – necessity sometimes dictates that we create courses like this.
2. Over valuing the approach
It can also be a mistake to index too heavily on an agency’s viewpoint or approach. Why? Because agencies can and often will describe their viewpoints in whatever terms seem to be resonating with the marketplace at the moment. Not only does this commoditize the approach (how helpful is it when everyone is saying essentially the same thing?), but there’s also nothing you can do to validate it. How do you know that a firm is really human-centered? How do you know they align with modern learners? The way an agency thinks about the work is meaningful – but only when the agency demonstrates that its people are capable of turning those sentiments into reality. If they can’t, it’s just talk.
3. Over valuing the process
Process descriptions, in my experience, are often of the least helpful in evaluating a firm. These are usually expressed in such an abstract way that they don’t address the biggest issues that will affect whether or not you reach the finish line. For example, how will the agency get the team, sponsor, and stakeholders all aligned? How will the agency get the subject matter they will need? How will the agency handle unexpected changes that come up? How will the agency communicate with the client? Trust me when I tell you – these issues will have a far larger impact on the success of the project than whether the firm uses ADDIE, SAM, or another special home-grown model.
4. Undervaluing team players
One time I hired an agency that put together an amazing, well-thought-out proposal. Once work began, we learned that everyone on the project team was a recently hired contractor, most of whom had never worked with the agency before. This begged the question, if I am impressed with an agency, its approach, its culture, etc., am I really getting that if the team they assign can’t live up to it, either because they are unfamiliar with it or are incapable? It was at this moment when I realized how important the project team is. In my opinion, the team who will work on your project is the number one thing affecting the success of the partnership. If I had to choose an agency based on only one factor, it would be the project team.
5. Undervaluing team organization
Each team will have its own micro-culture that is largely dictated by how the group is led. Thus, how a project team is organized has a big impact on the experience of working with them. For example, consider project leadership: If a project manager has authority over all of the other team members, timing and budget concerns will likely be the dominant factors when making instructional decisions. On the other hand, if an instructional designer is the lead, instructional efficacy will likely be more considered than timing and budget concerns. If you know how your team is being led, it can give you an idea of how they will approach working with you.
So, what should you pay attention to?
I’m not trying to argue that you shouldn’t evaluate portfolios, approaches, or process. You absolutely should. What I am saying is that you should regard these as relatively superficial factors that, while important, can mask the things that may have a much bigger impact on the success of your agency partnership. They should be used as a filtering tool in the early stages, but it would be risky to use them as the sole basis for your entire evaluation.
