Make e-Learning work: Outcome learning

Since I became CEO of easygenerator I wanted to have a clear vision and mission on what we do and why we do it. This led to my blogs on our vision and mission, ‘Output learning’ and ‘bringing learning to the workplace’. At easygenerator we use these ideas to give direction to product development and our market approach. Over the past year I joined these ideas, tried to develop them further and I renamed it to ‘outcome learning’.

Although I developed these ideas to give direction to easygenerator I believe that they apply in a broader context. I want to improve these ideas and by sharing them I hope to get some feedback. Therefore I decided to work them out in more detail in a series of post and I will give presentations about this subject at the DevLearn conference in Las Vegas in November and the Online Educa in Berlin in December.

Outcome learning
Outcome learning is an approach for learning in corporate environments. It can help organizations:

– to steer the learning processes differently,
– decide what kind of learning opportunities to offer
– to measure the effectiveness of learning
– make learning smarter
– connect learning to the day-to-day work.
– improve the ROI of learning efforts

It can help the e-Learning developer to:

– make e-Learning of higher quality
– align e-Learning more with business needs
– connect learning to the workplace
– get better insight in the results of e-Learning

It can help the learner:

– to set his own learning path
– shorten the time he spends on learning
– improve on his learning results

Starting points
I based this approach on the following starting points:

– e-Learning is ready to move to the next level.
– we must use learning objectives instead of page progress.
– we need to bring learning back to the workplace
– we need to tune learning more to the learners needs
– e-Learning development is a collaborative process

Three perspectives
The learning process looks very different depending on the view-point you take. Therefore I will describe it from the perspective of the manager, the e-Learning developer and the learner.

This post is part of a series of post on this subject:

  1. Make e-Learning work: Outcome learning (This post)
  2. Make e-Learning work: Outcome learning (2)
  3. Make e-Learning work: Outcome learning (3): The managers perspective
  4. Make e-Learning work: Outcome learning (4): The developers perspective
  5. Make e-Learning work: Outcome learning (5): The learners perspective
  6. Make e-Learning work: Outcome learning (6): The vendors perspective

Bringing learning to the workplace: Step 2 SpacedEd

At the learning solutions conference the opening keynote was delivered by John Medina with his presentation ‘Brain rules for learning’. The key note of his keynote is that it’s not about remembering, but more about not forgetting. If you have to learn facts, they must be repeated, or else you will forget. Therefore it is important to repeat new information regularly. This meade me think of a course I subscribed to in SpacedEd, it did just that.

SpacedEd is a service where you can create and follow courses. They consist only of questions and answers. If you enlist in a course you will get these questions in your mailbox. You will receive them in small amounts (typically 1 or 2 a day) on a regular schedule. They can be accessed through a PC or mobile device. After answering you will get an explanation on the question.

Spaceded course learning obout learning from Jay Cross

The questions you get will adapt based on your answers you gave. To improve retention, they will be repeated several times. If you answer a question wrong it will be repeated sooner. If you get it right one or more times in a row it will be retired from the course. Retire all questions and you have completed the course. The author of the course sets the spacing and repetition schema of the questions, as a learner you can adjust these settings to your liking.

I like this concept because it actually brings learning to the workplace, it’s adaptive and the repetition will improve retention. You can create and follow courses for free, but they also have a paid option. You should try it out. A course I can recommend is ‘Learning about Learning’ by Jay Cross.

This is my second post on bringing learning to the workplace. The first one was about Learning nuggets.

On demand: agile e-Learning development #LCBQ

Learning Cirquit Big Question blogThe Learning Circuits Big Question this month is: How do you address the “I want it now!” demand from stakeholders? We had a lot of discussion on this question but I would like to approach this months question from the perspective of the e-learning author. The on demand question has a big effect on them. Usually we use a waterfall method when developing e-Learning. The most used one is the ADDIE model, where development has five phases:Analysis, Design, Development, Implementation and Evaluation. It worked for years but it takes a long time to go through all the phases, not really suited for on demand responses. We need to get faster and more iterative.

In software development we have the same problem. We did e-Learning development through waterfall models for years, but now we have the agile approach. We do our software development at easygenerator through an agile method and I love it! I wrote a blog about it a short time ago. What you do is that you work in short sprints (two weeks) having a delivery at the end of a sprint. It will be a working product. This allows you to interact with your end users a lot more, get their input and take it with you in the next sprint. It really works. We build the new easygenerator version in 3 months. And not only it is on time, it has more (and some different things) than we planned for, it works and it is stable. So I thought this methodology could work for e-Learning content creation as well.

But as always it is hard to be original. Last week at the Learning solution conference there was a presentation on this subject and a I found a lot of interesting information on the subject. Therefore instead of enlightening you with my original work, I decided to give you a short overview of my most interesting findings.

Overview agile process (Image from the learning generalist)

Sumeet Moghe wrote a post in his blog ‘The Learning Generalist‘ on agile content development. He gives an overview on how it works and even ties it to action mapping. An excellent overview and he comments on it from is own experience. Great starting point.

Catherine Lang, and Jay Thayer of gave a presentation at the Learning Solutions conference in Orlando titled: Exploring the Agile Approach to e-Learning Development. They describe why they turned to an agile approach, how it works, lessons learned, tools to use and why it works for them. Interesting reading material.

An other interesting post can be found on the Integrated Learning Services blog, the author Jay Lambert is a strong defender of ADDIE. He wrote a blog on how to make ADDIE more agile by making it circular. It could be a starting point if you decide to explore this direction. In my experience with agile development it is very hard to be a bit agile, but it is a step in the right direction.

Antonio Guadagno & Matthew Tang recently presented in an online forum where they apply agile techniques they picked up from game development to e-learning. It also presents a very practical approach with lots of tips.

From my experience in software development and from reading I did I got convinced that this is an interesting approach to explore. Especially if you combine this with output learning to connect it to the workplace, learning nuggets to make it more adept and dynamic publishing principles. It will bring us closer to our goal: Bringing learning to the workplace.  We will hear much more of the agile approach in the future.

Learning objectives for formal e-Learning: Feedback requested!

At easygenerator we are in the finishing stages of our latest version. This is the first version I had a say in, so I’m quite exited, a real milestone. We started working on it at the beginning of the year. The plan was to finish it in April. The good news is that it is ready. We are setting some dot’s on the i’s and are doing some extra work. But it will be available before the first of April. Not only is it ready on time, I’m also very proud  of the functionality we have created. We will post a blog on our corporate blog with the details within two weeks.

This does mean that we are already preparing for the next version. We are in the stage where we need to decide what to do and what the priorities are. One of the subjects that is on the list are the Learning objectives. I’m not happy with the way we have implemented them in the current version. I addressed the topic of learning objectives in my blog on output learning and I tried to work from there.

When developing new functionality, we usually start with a document that describes the functionality in global terms. After some discussions we create user stories. They are estimated, decided on and then designed and build. Today I wrote the first draft document on learning objectives. Since I’m no specialist on that topic I’m looking for feedback. So I want to share this document with you and hopefully I will get a lot of feed back on it, so I can improve this. You can add your comment to this post or mail me.

Learning funnelGlobal objectives

Corporate learning always starts with a problem or a challenge. Based on the problem you have to define the ‘global learning’ objective. It can be something like: Being able to work with a new system, being able to apply new rules and regulations, know how to handle complaints, anything. The global learning objective is the flip side of the problem you try solve; it is the target you set for the course.

Before you can add more detail to your objectives you must have clarity on:

What does the learner need to know or to do and who needs to be able to do or know that. And finally is there a minimum knowledge of experience the learner has to have before he can start.

When you know the problem, your global learning objective, your target audience and your prerequisites you can define the detailed learning objectives. They are usually a split up of the global objective.

How could this be supported in easygenerator?

When you create a new course you can fill in the Learning objective form (optional). You describe the problem, the global learning objective, the target audience and the prerequisites.

Then you can add one or more detailed learning objectives. You can give a weight (a number) to each objective. You can indicate which score (per objective) needs to be achieved by the learner. If they score sufficient on all detailed learning objectives, they have passed. You can indicate whether or not you want this information published.

When published, a page will be created with the information from the form that you can include in the course. The learning objectives will be part of the scorm publication and the stand alone (HTML) publication.

Connecting the learning objectives to pages (content and questions)

After this you create the structure of your course with content pages, questions et cetera. On each page you can indicate to which detailed learning objective they relate, this can be more than one. They are presented in a list and you can check the relevant ones.  You can add a weight to each selected objective, the original weight points are also shown. The total assigned points will be added to the learning objective form. It is possible to add an objective from the page, it will also be added to the learning objective form.


In the learning objective form you can get an overview of assigned points per detailed learning objective. The total assigned points are shown behind the original weight points. If they don’t match the author can either change the weight points on a course level or on the page level. After this the author can create the content for the course.

I’m really curious what you think about this, does it work for you, or should we do it in an other way. I would be very happy if you want to share your thoughts with me.

Bringing learning to the workplace: step one ‘learning nuggets’

In my posts on output learning I have tried to describe a method to connect learning to problems in the workplace. In this post I want to explore a first concrete step towards that goal.

I think that the first step could be formed by using ‘learning nuggets’. e-Learning is now often delivered as a SCORM package in an LMS covering a complete subject. This is fine for a first introduction into that subject, but research shows that just a small percentage of the presented knowledge is retained. The major reason is that you often can’t apply the learning content immediately into practice while you are learning and that is the most important condition for retaining new knowledge.

The solution is to bring the learning to the workplace. In order to do that we have to alter our Learning content. In stead of large subject oriented courses we need switch to small task oriented learning nuggets. And that is not the only thing. The worker has to be able to find that piece of content. You can offer search facilities to the worker, but it would be even better to ‘read’ his working context. When a task is performed using software there is a possibility to connect the learning nuggets to the context-sensitive help. When a user presses F1 we can offer him not only support for the use of the system but also additional information (nuggets) with task support.

When new software is introduced this is often supported by e-Learning, most of the times by a simulation where people can learn how to perform their designated tasks in the new system. This is a good way to introduce the new system, but it is not connected to the working environment. We could reuse that same content and split it up into small learning nuggets, based on tasks or sub-tasks. This is very easy to do in authoring systems like easygenerator. Then you publish the learning nugget as HTML package and place it on your intranet or on an internal server. You could enable the end user to find these nuggets through a search engine or even a simple list. But it is also possible to add links to these nuggets in your context-sensitive help system. When a user is performing a task in the system and he needs support, he presses F1. A list with relevant subjects (including the nuggets) is presented. The user clicks the link, goes through the learning content, returns to his system and completes the task. It seems to me that this is a very easy and cheap way to connect the learning to the workplace in a different way.

Impact of informal learning: output learning #LCBQ

The question of the month #LCBQ is:

How do you assess whether your informal learning, social learning, continuous learning and performance support initiatives have the desired impact or if they achieve the desired results?

All these terms have one common denominator; they all are non-formal learning activities and processes. And managers (being one myself  for more than a decade) tend to get nervous by these kind of activities. You told that you have to do these kind of things, but the point is that you have to invest time and money into them to make it work. But at the end of the day you don’t have a clue if whether it was worth it. So where is your production improvement and your return on investment. How do you measure it? So this is an excellent question.

With formal learning (classroom and LMS based e-Learning) you usually get an assessment in the end, if you score enough points you get your certificate. You can ask yourself if this is a result that counts; does it affect the workplace in a positive way? It only will be if it has an effect on the learners working skills, making them more effective and productive. 40 employees with a certificate do not guarantee that. So in fact you can ask the same question for formal learning activities. The point is that we got used to this form of learning and measuring.

The what: Output learning

The output of learning is not what you have learned, but how you can apply it in your everyday work. Therefore if we want to measure the impact of learning (formal or informal) we need to measure that impact.

Recently I wrote a blog about ‘output learning‘, based on a management theory called ‘output management’. In a nutshell it works like this. You encounter a problem in the workspace , then you set your learning objectives (that lead to tackling the negative effect of the problem), determine the requirements that set the boundaries  for that solution and then the worker/learner is free to solve his problem anyway he wants, as long as he stays within the boundaries set by the requirements. This concept is more for formal learning activities and here is more to the concept, but we can get two things from it that are relevant to the question.

Learning funnel

If you are able as a manager to apply this model to informal learning you will get more grip on the informal learning processes, while letting the worker free in choosing his own way to the solution of the problem. You could create a list with a certain types of problems that workers can solve with informal learning, or you can set how much time they are allowed to spent. By outlining these requirements like these beforehand you prevent problems and you give your workers clarity on your expectations.

The other thing is that the output of learning is not the learning on itself but the effect it has on the workplace (you want to improve efficiency and productivity). So, did the learning activity solve the problem within the given requirements?

The how

So how do you  measure this impact of informal learning in problem solving at the workplace? This isn’t an easy question and there is no easy answer (or in any case not one answer). Here are some possibilities:

Log. A solution would be having your worker keeping a log, with problem, activities, time spent and the outcome. It would be a great source of information but is not likely that it will be successful; it is to much work. But if someone was active in an informal learning environment you could present him with one or two questions in the end, actively gathering information and feed back from the worker.

Survey is an other way of gathering  information and feed back from your workers, you should do this probably once every quarter.

Measure the use of content and the activity in networks. You can measure the activity in all kind of networks and environments. There are all kinds of tools available like Google analytics that can give you a great insight in which content is used, how often it is used and who uses it. Especially in combination with a survey and an analysis of user generated content this will provide you a with a lot of useful information on the informal learning activities.

User generated input . I believe that this is the most valuable one because the workers actually can benefit from it to. We do this it in Twitter, Yammer, Facebook, Blogs and all other social media, we don’t do it enough in a corporate environment. Enabling this is not that hard:

1. Make it a rule that whenever someone can improve information he has to leave a comment.

2. Give him the possibility to like or dislike certain information or can indicate if the information was useful.

3. If someone has new information (a new FAQ item or best practice) he has to add that to the system.

You will have to moderate this, but it will give you information on what works and what doesn’t work and you will capture the knowledge of your workers and give it back to them.

From output management and e-Learning to output learning

Recently I have reviewed a lot of e-Learning courses to get ideas for our further product development of easygenerator. One of the things I noticed was that learning objectives often play a secondary role or no role at all. Most of the time they are only anchors to attach scores to, so you can measure if somebody has successfully completed your course. In easygenerator learning objectives are present, but they don’t have a prominent place. I was quite surprised about that. Originally I was a teacher for economics and social studies and especially for social studies we often created our own lessons and learning material. The first thing they taught me when I was training to become a teacher was that you always start with your learning objectives. The other thing that I noticed is that sometimes it wasn’t clear at all how the course and objectives relate to the working place or life problems; often it isn’t clear what problem the learning wants to solve.

Then it suddenly struck me that you could match the principles of output management to (e-)learning. There is a great book on about output management by Filip Vandendriessche, which is my ‘management bible’.  I wrote a blog about output management some time ago. With this post I try to apply his output principles to learning. If you want more information on the original, you can read my blog, go to the output management site or buy his book (which I really recommend).

The Learning Funnel

Filip uses a the term ‘management funnel’, it goes from problem, goal to requirements and finally to solutions. To alter this into a learning funnel I have changed ‘goals’ into ‘learning objectives’ and ‘solutions’ into ‘adaptive learning’.  This results in the following:

Learning funnel

The funnel is a funnel because the manager/teacher is authoritarian on the strategic level,  less authoritarian on a tactical level and must ‘let go’ on the operational level.

Strategic level

Motivation is an important precondition to learning. People are often motivated by their natural curiosity and their need to develop. In the corporate world this is not always the case, we have to motivate the workers to become learners, they often need a reason for wanting to learn something. In this case the start of learning can  be a problem that they encountered during work. When you are not able to do a certain task, either through lack of knowledge or lack of skills or anything else that is a problem. The first step is that people need to admit that they have a problem. After this we can take the next step. The problem might be solved by having better equipment, but a lot of the times the solution will involve a learning process. When learning is the answer you need to define learning objectives. They are the flip side of the problem, it is the desired output in terms of learning needs. Learning objectives are often a set of sub-objectives, written in more detail than the problem. Learning objectives must make sense to the learner: “I need to be able to do …. in order to achieve …. “. Measuring success is easy: You need to be able to apply what you have learned in practice and thus solve the problem.

When people accept that they have a problem and they do accept the learning objectives they are motivated to solve the problem, it has become their problem. Motivation is one of the most essential parts of learning. Both in corporate learning and in our educational system we often forget this. It’s better to motivate your learners than to control them!

Tactical level

The next step is to set requirements. As a manager/teacher you do not only want the problem to be solved, you want to set certain boundaries in reaching that solution. Requirements can deal with time, budget, quality, support et cetera. On the tactile level you are less authoritarian than on the strategic level. Certain negotiations might be necessary. The requirements need to be acceptable to both manager/teacher and learner. When a learner does accept the problem, the learning objectives and the requirements he will be committed to solve the problem within the set boundaries.

Operational level

Because the learner is now committed and motivated all solutions to the problem the manager can lay back.  All solutions are acceptable as long as they meet the requirements and solve the problem. The learner is free in choosing his way of solving the problem and reaching the learning objectives. There are many ways to acquire the desired knowledge, skills or insights. We offer often just one way of learning (a course or a class) but we need to facilitate more learning paths with very different solutions. It might be the best solution to have a skilled person show the learner how it works, and all we need to do is connect those two people.

Motivation pyramid

In Filips theory this is called conflict pyramid. The chances to have a conflict on an operational level are far bigger than on a strategic level. There are always more solutions (learning paths) to one problem, and it is arbitrary which one is best. On a strategic level the chances of a conflict are way smaller. When you can’t execute a designated task, you have a problem, it is that simple. I renamed the ‘conflict pyramid’ to ‘motivation pyramid’ because avoiding conflicts is not enough for learning, you need to motivate the learner.

Motivation pyramid


If you place the learning funnel and the motivation pyramid in one image you will get:

Learning funnel and motivation pyramid

The essence of this is that motivation will increase when a learner can create his own (learning) path to the solution. Because the boundaries are given the manager/teacher can let go. The proof of the pudding is in the eating, so does the solution the learner came up with, solves the problem within the given requirements.

Writing this post created some new insights for me. I have to develop these ideas further but I think they can help in steering diffuse learning processes like social learning and informal learning and it does give more direction to adaptive learning then just ‘let go’. I was looking for a new term to capture our activities in the learning field; the term e-learning becomes outdated. ‘Output learning’ might be an option. I like that it describes the content and the process instead the medium like e-Learning does.

Note: The concept of ‘output management’ is created by Filip VandenDriessche. The theory and images are a reworking of the originals from his book ‘de input-output manager (ISBN13 9789077432037)’. This post was published with the permission of Filip VandenDriessche. The book is available in three languages:

English: Leading without commanding (Published in-house), French: Diriger sans imposer (Eyrolles Paris 2008) and the Dutch version, Leidinggeven zonder bevelen (Lannoo 2011).


Get every new post delivered to your Inbox.

Join 1,095 other followers

%d bloggers like this: