Finding the right elearning vendor can be extraordinarily difficult.
As a buyer, you’re exposed to a tantalizing veneer put up by the sales and marketing team that keeps you from peeking behind the curtain and getting a better idea of what you will actually get. I know because I’ve been on both sides of the table – as a buyer and a seller. I’ve seen the mistakes that buyers make in the hiring process and I’ve made plenty of them myself. I’ve seen the problems that can come up once a project gets underway and how some of them could have been avoided in hiring. Through all of this, I’ve identified some key questions that, when asked, can go a long way toward making sure you have a good experience with your learning vendor.
Key questions to ask your learning agency
In the process of hiring an agency, it is essential to try and ascertain, to the greatest extent possible, how the agency will manage critical factors that affect partnership success. (Not sure what those are? Check out this blog post.) To do that, I recommend exploring the lines of inquiry below. If at all possible, I would try to ask these questions of someone from their delivery teams (rather than just the sales staff) since they are in the best position to give you the specifics you may need.
What you are looking for in their answers will depend on your situation, what you value, how you prefer to work, and the constraints you are facing. For each question, I’ve offered what I would be looking for given my own preferences, perspective, and experience in the field, which may or may not perfectly align with where you are coming from.
1. Can you meet us where we are?
I would never ask the question in this way, but the point is to figure out what the typical starting point for them is and to see how that lines up with where you are in the process. Do they expect you to hand them a neat outline? If so, do you have one? If not, can they work without one? If you need help exploring the problem a little and further clarifying the goals, do they do that? If so, can they tell you about recent projects where they have? Is there a step in their process that accommodates this, or will they be improvising?
What I would be looking for: I would be looking for evidence that they are not so in love with their own process that they cannot flex it to suit almost any situation. I would be looking for evidence that, as a matter of standard procedure, they take the time to deeply explore the problems you are trying to solve, why it is important to the business, and what may be hard for your learner populations. In other words, they don’t just leap directly into trying to make something.
2. What will I need to provide you for you to get started?
This is similar to the first line of inquiry but different in a subtle but important way. It is rare that we start projects with subject matter that is documented in a single place that is neat, well organized, and complete. It usually is scattered amongst a bunch of PowerPoints, PDFs, and in the heads of various subject matter experts. Try to find out what they typically receive from clients, how that aligns with what you have, and if not, whether they are able to adapt.
What I would be looking for: My own perspective on such matters is that the less I have to do before engaging an agency, the more I can concentrate on other things. In this context, that means giving them what I have in its current form and letting them sort it out. I happen to think this is one of the main value-adds that a good instructional designer can offer. But the bottom line is that I don’t want to have to gather everything together and write a neat little essay about it before they can get going.
3. What work will be required of me during this process?
In this case “me” refers not just to you, the sponsor, but to anyone in the buyer’s firm who will have to spend time on the project. At a minimum, someone from the client will have to review deliverables, coordinate access to subject matter experts and stakeholders, etc. Will you or someone at the firm have to come up with ideas for instructional strategies? Write content? Create storyboards? Even produce PowerPoints?
What I would be looking for: See my comment for the question above. I will have enough work coordinating reviews, aligning stakeholders, and overseeing the project without also having to do the writing (or have subject matter experts do it). Besides, instructional design by definition is the act of identifying optimal learning activities and creating the instruction. If someone on my end has to do that, what am I actually paying for? I might be better off hiring an offshore agency just for the production (i.e. graphic design, media development, and programming – which involve almost no instructional design).
4. How will my project be staffed?
For me, this is the most important line of questioning. In fact, it’s really several lines of questioning combined into one:
- What roles will there be on my project team?
- How will team members be assigned to the team?
- Which roles, if any, will be W2 employees, and which will be 1099 contractors?
- Which of those roles will have final decision-making authority?
- What is the decision-maker’s background (project management, instructional design, etc.)?
- How long has that person been with the company?
- What position did that person have when they started with the company?
- Has that person worked on projects like mine?
Depending on the stage of evaluation, the agency may not be able to tell you the specific people who will be working on your team (though if you get to the proposal stage, I would try and pin them down). In that case, just ask about such things in general, but learn as much as you possibly can, as this will have the biggest impact on your project hands down.
What I would be looking for: I would look for a project lead who is a full-time W2 employee, has been with the company for a long time – preferably having moved up into the role rather than having been hired from outside – who has run many projects similar to mine, and who has a background in instructional design. I would have concerns if the project lead were new to the company, new to the role, or had a background in project management, graphic design, or frankly, anything other than instructional design. If the project lead were a contractor, it would be a no-go for me. I would want to make sure that the project lead had full authority on the team. For a role like instructional designer, I would primarily be looking for a W2. I would look for a project manager but one who is not the leader of the team. For graphic design and course development, 1099 would be acceptable but not preferable. Overall, if the team has experience working together, that is a major plus.
5. How do you manage mid-project changes?
This line of inquiry is all about discovering whether they have a specific approach to dealing with changes and, if so, what that is. This may require some digging because you’re really trying to answer a few questions:
- What is their attitude about changes? In an ideal world, are there no changes?
- How much discretion will you, the sponsor, have to alter course at various points?
- How set are they on the current plan and course of action?
- What constitutes a big change and a small change to them?
- In their view, what sort of changes cause problems and which do not?
- How do they deal with the small problems?
- How do they deal with the bigger problems?
- What sort of things would cause a scope change and how does that process work?
What I would be looking for: Changes are an inevitable part of every single project. When you begin a project, neither you nor the agency have all the information needed to make every decision on the project. Yet in order to come to a contractual agreement, it was necessary to make a bunch of mutually agreeable assumptions. Once you get into a project, more information will be revealed and you all will be able to make more informed decisions. That’s why changes are a good thing and as a sponsor, you want at least some degree of freedom to make changes. Yet many elearning teams, particularly those run by project managers, view changes as bad and to be avoided. Don’t get me wrong – we can’t just be changing everything willy nilly over and over again. But there should be room given to adjust as new things emerge. With all of that said, from my perspective, the ideal way of dealing with changes that could affect scope is to approach the issue with a “let’s see how we can get this done” attitude. In practice, this means first exploring whether you can take from some other part of the solution in order to accommodate the new one. I would be looking for a firm with this attitude and one that can be flexible in their process, particularly in the upfront stages when we’re still clarifying goals and intent.
Bonus questions:
I’m feeling generous today so let’s add three additional questions that could be helpful.
6. What tools will you be using?
Every elearning agency uses tools to create its solutions. On the one hand, it shouldn’t matter what tool the agency uses as long as you get a solution that you like. On the other, if they use a proprietary tool or one that is very niche, that may make it difficult to update in the future – or even impossible (when a vendor does this, it’s almost like they are licensing your content back to you, because you have no ability to do anything with it without paying them more). If the tool is proprietary, that means you will have no choice but to work with the same agency if you need to make updates. If the tool is from a niche provider, the tool may not exist in the future but even if it does, it may be hard to find people who are skilled using that tool. If you are not planning to make updates in the future, this may not be a concern, but it is helpful to know where your agency stands on this issue.
What I would be looking for: For interactive elearning, I’d look for an agency that uses Articulate’s Storyline (or any Articulate tools). It’s not because I love Storyline all that much (believe me, I don’t). It’s the fact that it has become standard, has been around for a long time, and is likely to be around for a long time to come. Courses built in Storyline will work with virtually any LMS and it’s easy to find people who can build in Storyline. I’m open to other tools but Storyline is the default in my book.
7. Do you use templates and if so, how do you use them?
Some elearning agencies use templates in their work and I’m not just talking about graphic design templates. Some have a menu of instructional activities that you choose from (e.g. choice “a” is a role play, choice “b” is a drag and drop exercise, etc.) Some agencies can be so templated that all you basically do is draft some text and drop it into the template and “shazam” – instant elearning. Templates reduce the time and cost of producing elearning. The downside is that they limit how custom your solution is and can leave you with instruction that doesn’t always fit the learning objective. It’s up to you to decide how much you value cost versus value. But regardless of where you fall, you should absolutely be aware of how your elearning agency works in this regard.
What I would be looking for: I strongly believe that if you want a person to learn from your instruction, the teaching strategy has to closely align with the specific learning objective. I would never hire a firm that uses templated instructional activities. I would honestly rather not hire a firm at all and find some other way of accomplishing the goal – or do nothing. On the production side (i.e. the visuals, the development, the media), I could accept templates if it meant the reduction in cost was sufficient, but I typically prefer a custom approach.
8. Do you use AI and if so, how do you use it?
AI has introduced some new factors that need to be managed if you are either an elearning firm or someone who hires them. Three big ones have to do with attribution, accuracy, and quality. Regarding attribution, when you use AI to generate content from open sources, it is pulling from a broad set of data, some of which may be proprietary and others of which may be of suspect credibility. When it comes to accuracy, we all know that AI engines often produce inaccurate and even fabricated data as well. Finally, there’s the issue of quality. AI models lean on the internet for answers which means the data it provides is only as good as the data on the internet. For these reasons (and others) many organizations who use AI in the construction of content let whoever uses the content know. As the buyer, it is very important to understand how your elearning agency intends to use AI on your project.
What I would be looking for: Since the content of the learning program would most likely be specific to my company, I would be trying to make sure that they are not leaning too heavily on AI to generate content. When they do use it to generate content, I would be particular about what sort of content they use it for. (AI could be great for topics related to, say, the biology of diabetes. I would be exceptionally cautious about depending on AI for things like coaching, or any sort of soft skills.) I would want to be sure they are properly attributing sources when using AI. In general, I would be most comfortable if they were using AI in small spots throughout the project to save time (e.g. feeding some content into an AI model to get an initial list of assessment questions) and not leaning on it too heavily to either generate content based on data from the internet.
When it comes to hiring a elearning firm that will be a good fit for you and your company, there really isn’t a silver bullet that will ensure you pick a winner every time. But I’m quite confident that if you properly explore these issues, you’ll greatly increase your chances.
