learning objectives: you need feed back to make them work

If you read my blog on a frequent basis you will know that I believe that Learning Objectives and learning are two sides of the same medal. I want to share a story with you that gave me an insight yesterday.

Some time ago I wanted to improve my bathroom and in a moment of insanity I decided that I would do that myself. It worked out sort of OK, but every now and then the bathroom would leak. Because it is situated on the second floor, I have a problem on the first floor as well. This was not a part of my master plan. So I needed to have a contractor that could fix that for me. I hired Albert, a local contractor, that did some fine work for us a while ago. So last week he was in my house for a few days fixing the problem. I work 3 days a week from home, so he saw me sitting in my office; mailing, conferencing, calling (over VOIP) and all the other stuff you do when you work. He was really amazed that all this was possible. He told me that he has a website (made by a local guy) and that that guy also installed a PC with internet connection and mail in his home three years ago. He never touched it since, because he hasn’t got a clue what to do with it.

Later on he decided that I would need a new shower base (if that is the proper word for it; anyway I mean the thing you stand in while taking a shower), and he told me that it would cost me between 350 and 400 Euro. I surfed the web and found the perfect one in 5 minutes for 130 euro, I ordered it and paid for it on-line. He was in awe and understood that it would be profitable for his business if he would be able do that as well. I offered him to teach this. So we set out to define the learning objectives for this, they became (with the action mapping rules in mind):

  1. Albert is able to send and answer mail from his own PC
  2. Albert is able to search ‘Marktplaats’ (a Dutch eBay) and buy things from it

They looked simple enough. Yesterday evening I went over to his house and we started. It turned out he did really know nothing about computers at all. I had to explain that you don’t only have to place your mouse on the desired position, but that you have to do a left mouse click as well. When he typed a few words he asked me ‘How can I get a gap between the words?’ and I had to explain the function of space bar to him.

spacebar 3

Based on his feed back I had to adapt our learning objectives. They became:

  1. Albert knows how to operate the main function of his PC
  2. Albert understands the difference between mail and internet
  3. Albert is able to sent and answer mail from his own PC

We went on from there and decided that we would need several session to reach the original objectives. For me this was a learning moment. We changed not only the learning content but the learning objectives as well. With easygenerator we are proud on our adaptive courses that will advise a learner on an individual basis; but only on a content level. I now realize we have to offer facilities on the level of learning objectives as well. Another two hours well spent.

Returning to the didactical roots: innovation in eLearning?

Earlier this month I presented at DevLearn on connecting learning to the business and this week I did a webinar and a seminar on adaptive learning. During these sessions I noticed that our basic approach (Determine learning objectives, Figure out how to assess and then create only the content that is really needed) is far from standard.  Most people create content, create an assessment and that is it. But the funny thing is that this ‘old school’ approach is the foundation of innovation at easygenerator.

Originally I’m a teacher in social studies and economics. They taught me that for every lesson you want to create you need to figure out your goal first and that you need to find a way to asses if that goal is reached in the end. Only then you could start creating your lessons. I did apply this approach through my working live: with teaching, with writing books (on bookkeeping – how boring can you get?-), when I create eLearning and even when I manage a company. I know it is not common practice, but I still believe that this is the way to go.

Old school didactics
Let’s first take a look at this old school approach.


As said you start out with your learning objectives. Creating sound and useful objectives is an art in its own right. I will not go in too much detail here but I’m a fan of the action mapping approach from Cathy Moore. The essence of this approach is that learning is not about obtaining knowledge but to (learn) to be able to perform a task. Cathy doesn’t link this to learning objectives, but if you do, they should state what the learner needs to be able to do.

The second step in the development process is the assessment: how do you prove that the learner is able to do the task? You can do this by asking questions, presenting cases, really anything that will measure the performance and comes up with a score. By the way thanks to our new emerging standard (‘Tincan API’ aka ‘the experience API’) we will be able to measure this in real live and use the outcome in an eLearning course). When you create good cases (or scenario’s) this assessment will be the learning experience by itself.

And only then you start creating the content. But in the spirit of Cathy Moore only the content that is really, really needed to (learn to) do the task. When in doubt leave it away, ‘less is better’ and much cheaper!

We have applied this principle in the authoring platform of easygenerator and it has become the foundation underneath the innovations we have created and will create in the future. I will explain.

In easygenerator we created a dashboard to create and manage your learning objectives. You can’t create a course without a learning objective (if there no goal there is no point in creating a course after all) in easygenerator.

After creating the course you need to set how to measure the progress in the course. You do that by connecting the Learning objectives to questions and cases. In fact you are determining how to assess the objectives. Finally you connect these questions to related information pages.

And this simple approach will change and enable a lot:

  1. It will change your design process and with that the kind of course you create.
  2. The learner is able to see the objectives and his progress on the objectives during the course.
  3. The course is able to present a personal study advice to the learner.
  4. You will be able to report the outcome per learner per learning objective, giving you meaningful data to evaluate you course and your contribution to the companies goals.

These are only the first developments we did based on this approach, a lot more will follow. This video shows you how this works for the learner and for the author.

Based on these very basic dialectical principles we will continue the innovation of eLearning courses and the creation process. Some of the things on our road map are:

  • Create non-hierarchical metaphors and interfaces for eLearning courses (no book metaphor).
  • Create better support for designing eLearning courses in our authoring environment.
  • Implement TinCan
  • Create learning maps, where the learner can navigate through on his journey to reaching his learning objective
  • Create better support for case based and scenario based eLearning in the authoring environment

And there will be much more. But the bottom-line is that this idea is independent of a tool, it is how you organize your development process. You can do this on paper if you want, but I believe eLearning developers should do this much more, regardless of the tool they are using.

How to keep formal e-Learning relevant

We all know that e-learning is changing, we all know that our learners have changed. The rise of the internet, social media and mobile devices have changed our world. It turned out that it is much easier for a learner to adapt to these changes than for a e-Learning manager or developer. Over the past 16 months I have written all kind of post researching this change. I was recently asked to present on this subject in a webinar. In my preparation I went through all the posts and was for the first time able to merge them in a coherent way. I wanted to share this presentation with you.

Additional information on a lot of the subjects that are in the presentation I wrote about earlier. These post contain a lot of links to other resources on the internet on these subjects:

•Output management
•Agile development for Software, and for e-Learning
•Learning metaphors, learning maps
•Outcome learning (series of posts)

You can attend the webinar if you like (Wednesday February 15th 2012 10.00 am and 2.00 pm EST). See for details the site of Interactive Advantage.

A new metaphor for e-Learning: learning maps

Yesterday I visited the Fraunhofer institute in Karlsruhe, Germany. They have been investigating a learning map metaphor for the past few years. Interesting visit. So what did I learn from it?

The Fraunhofer institute is an impressive organization 18.000 employees working in applied science. So they approached this research in a scientific way. They use web didactics as a didactical framework and build a very complete solution around that. If we take a closer look at their solutions there are some things that I like, but also some things that need improvement.

Image of the prototype from the Fraunhofer solutions for learning maps

Image of the prototype from the Fraunhofer solutions for learning maps

If you look at the image you will see a course that has several learning paths. Based upon the results they use a branching mechanism to steer you in a certain direction and they will choose the proper path for you. This I don’t like. The purpose of a map is to give insight to the learner in the content and the possible learning paths. There is no point in presenting alternative paths if you can’t take them. What I do like is the fact that you can see the route you have already taken (dark on the image) and that they give feedback on the branching. At the first intersection you can see in the map that you will navigate to the top of the map if you score over 50% and that you will take the bottom route if you score lower. The biggest problem is the amount of information. You can zoom in and out of a map. On a high level it is OK, it shows the routes and it presents the learning content as a smaller or larger town. When you zoom in (like in the picture above), there is way to much information. Each symbol is a combination of media type, content type and level. This combination makes it very too complex. As always the hardest part is leaving things out in order to make it simple and intuitive. An other problem is the representation of learning objectives. They are presented on the top of the screen in a sequence. The selected learning objective is indicated with a red square. This way you don’t have an overview of all objectives and you can’t see the relations between the objectives.

It is easy to have comments on the work of somebody else, but it is way harder to come up with suggestions that make it better. I tried to come up with them in the form of requirements for a new metaphor. Here they are:

  • The new metaphor should give the learner an overview of all learning objectives and their inter relations in one simple overview.
  • Information must be shown to suggest to best learning path, but the learner should be able to decide which path to take.
  • The metaphor should show the followed learning path.
  • Information should be dosed carefully, so we don’t want to present all information at once. You should be able to zoom in to one specific area only.
  • We need to have simplified icons to inform the learner about the content that is available.
  • Creating a learning presentation based on a metaphor should be as easy as building a course with a book metaphor, It should be automatically generated, no scripting required.

I do believe that maps are a potential candidate for a new metaphor, the challenge will be to make it simple, intuitive on the one hand but rich on the other. The first post on a new learning metaphor was very well read (it is already my best read post ever) and I got a lot of comments. Thanks for that, I got great input that really helps me. Apparently this subject is of interest to many people and I will definitely follow-up with more posts.

See also:

DevLearn Day 3: the final day of a great week

I started the day with a Morning Buzz session about management and leadership by Michelle Fanfarillo and Bill Harisson. They are from Intel and told us how they try to connect to the business. At Intel learning is a support organization (as always) resorting under HR so they have to work hard to connect to the business. Intel is huge, 96000 people, 200 plus countries, 15 main sites, 10.000 managers. What they did was to give each HR director of a business group a learning consultant. They also have country organizations with a HR director, they got a learning consultant too. They now are setting up a steering committee in order to have a central point of direction.

This is how they manage their managers. I asked them how they manage the increasing needs of individual learners and they told me that they haven’t figured that out yet. The conclusion was don’t work from learning goals but from organization/business goals. Insure access to the business leaders and use a steering committee.

Internet time alliance, Interactive review of DevLearn
The second session I attended was run by the Internet time alliance. All five of them where there. They did a review of the conference results. They gave their findings, showed interviews with conference attendees and asked the people that attended the session. I made a mind map of it:

But the most important insight was the curation of content.

After this I had a final meeting with my colleagues Chris and Steven, they both had to leave.

I had some meetings in the afternoon. One with David Holcombe (president and CEO of the Elaerning guild) and Heidi Fisk (Executive director) about the possibility of an European Guild conference. Conclusion it will happen, the question is when.

But by far the most exiting meeting was with some people of LINGOs. They now support Ngo’s who work in developing countries. Eric Berg came up with a plan to up the stakes a bit, the new goal is to make knowledge and learning available to anyone in developing countries. I just checked how many people live in developing countries and found the following figure: 5,727,771,964 (give or take a few). This is a mind-blowing initiative. If they pull it of it would be worthy of a Nobel Prize. Imagine if all people have access to learning and they could really develop them selves. It would mean less poverty, less hunger, less war, fewer people dying, more people having to chance to a better life. They formed a steering committee that will work out the plan in further detail and Eric has asked me to be part of that committee. I’m very honored that I can be a part of this great plan and I hope that I can contribute to it. I will be writing more about this initiative in the future. Wow!

I have five hours left before I have to pack my bags and go home. I will try to get some sleep before that. For now preliminary conclusion of DevLearn is: it was a great week!

Devlearn conference day 2: curation

I started with a morning buzz session with Jay Cross. He told us how he deals with the information overload that is flooding all of us. The message is simple. You go from push to pull, with RSS feeds you pull information in instead for searching it on the web. The second step is that he uses software (Aggregage) to store, organize and publish the information he pulls in. This way he creates a place where people can go to see his selection of information for certain topics. This means that people are replacing search engines. Instead of looking things up with google you plugin to the knowledge of somebody who is an acknowledged specialist in his field. Jay does this on workingsmarterdaily.com. Another example is the Elearninglearning the site of Tony Karrer. This blog and the easygenerator blog are ‘republished’ or aggregated through that site.

And that covers the keynote of the day too. Steven Rosenbaum presented ‘the future of learning is context’ and his story was an exact copy of Jay’s. Steven calls this ‘curation’ and told us that by publishing (like this blog or a twitter feed) we are all curating information. Funny coincidence to have those two session back to back with exactly the same message.

After the keynote the expo demanded our attention, I wasn’t able to go to other sessions. That is the flip side of being an exhibitor, you are there for others and not for yourself. I wished I was able to clone myself and do both.

The expo has been very good for us we have a ton of leads en the coming weeks will be packed with following up, giving webinars and assisting people who signed up for the 30 day trial. It’s great, it is why we are here.

I also presented on the ‘Emerging technology stage’ together with Marten du Prez of aNewSpring. I showed how you can create individual learning paths within an e-Learning course, he explained how they make it possible to create individual learning paths between courses.  I told how frustrated I got when I was creating e-learning courses. You work from learning objectives and design a course based on that. Then you go into your authoring tool that doesn’t have any support for learning objectives and you loose it all. In stead you end up with a powerpoint like course that reports based on page progress. It turns out , frustration is shared by almost every developer. I compared page progress with giving a car key to somebody and telling them that they have to stick it in the contact of the car for 10 hours and then they will get their drivers license. You don’t do that, why should we approach e-learning like that. By the response I received it seems that we have found a working solution for this problem and that we are adding real value for both the developer and the learner. Michael Allen, the CEO of Allen interactions came over to me in a bar and told me that he loved our approach of learning objectives. That’s another sign that we are on track.

The rest of the day was filled with meetings and working at the booth. I had lunch with Eric Berg, he is the Executive director of Lingos. They are a great organization that supports ngo’s with e-Learning facilities and knowledge, they really make a difference with their work and I’m a fan. Easygenerator is a sponsor of LINGOs, offering easygenerator for free to all associated ngo’s. On a personal title I’m an ambassador for them, trying to gain more support in Europe. I will represent LINGOs at the online Educa. It’s always an inspiration the meet Eric and I’m happy that I can make a contribution to his organization.

We had a quit evening dinner with our team and evaluated the expo and made plans for how to follow all those leads in the best possible way. For us DevLearn is a huge success and we will be back next year at the expo.Yesterday was the last day of the expo so I will have some time to attend sessions today. That and some meetings will fill the last day of the conference.

DevLearn conference Day 1: a hectic day

The actual start of the conference and what a day it turned out to be. It began early at 02.00. I woke up after just two hours of sleep (the jet lag hit me) and I was wide awake. At three I decided to go down to the casino with $100 in my pocket and I played black jack for half an hour. And it paid off.

I’m not a gambler at all. It is my second time in Vegas and it was my second time in a casino. The previous time I won $1200, so it is Las Vegas 0 – Kasper 1700! I went back to my room and did some work, at five I discovered that my colleague Chris was awake to. We went for some coffee and went to the gym.

Then I went to my first session of the day. At Guild conferences you have so-called Morning buzz sessions, they start at 07.15. Crazy initiative, but it works. I went to the session of Charles Jennings in the management track. It was about governance, how do you connect your learning department to the business. His main point was that you should have a sort of board who makes decisions about the ‘What’ of the learning. He told about his experience as a CLO at Reuters where they had a board with business leaders, HR and Learning. He told that he did some research and found that about 50% of the companies have boards like that, but only 12,5% of all companies had business stakeholders in that board. His point is that you should have them. We discussed about the experiences of other attendees and every interesting point was made by Jay Cross. He said that you have to do a performance analysis instead of a training need analysis. That way you can tie the learning better to the business. Sounds logical to me.

Then it was time for the opening and the keynote by Michio Kaku, I had really high expectations because the keynotes I attended at the guild conference in Orlando where of an exceptional high level. He disappointed me. Michio Kaku is a theoretical physicist and basically he told us that computers would become so small in 20 years time and so cheap that they would be everywhere (but invisible) and completely change our live. Instead of computers you will have contact lenses that will connect you to the internet and project relevant information into the real world (augmented reality). His talk didn’t bring me any new insights. Clarck Quin made a mindmap of this presentation, so you can check that out.

After the keynote the expo opened and we where flooded by people who wanted to know more about easygenerator. This interest continued until the expo closed (at 7.00), we worked our stand with the four of us and it was hectic. We got great responses and great interest in easygenerator. Hopefully this will lead to loads of new customers.

At 9.30 Bill Brandon published his article about  easygenerator in the Learning Solutions Magazine, great stuff. I will let him interview me any time he wants.

At 12.00 we had a product presentation at the big stage in the expo. It was supposed to be presented by my colleague Chris and Dan Richards (of our US partner Advantage Interactive). But yesterday evening Dan became ill and had to go home. We decided that we would improvise and I got on stage and did a demo of easygenerator. I think it worked out OK.

At 2.45 I had my session on Outcome Learning, there where about 35 people in the audience and I really enjoyed sharing my ideas. It was the first time I presented about this in public and I got good responses. All sessions are evaluated by the audience, it will be interesting to see how I actually did in their eyes. I tried to embed the Prezi, but I didn’t succeed. So here is the link to it.

It was so busy at the stand that I completely missed the second keynote of the day by Tom Koulopoulos. The title was Living and Learning in the cloud. I’m sorry that I missed it. Clarck made another mind map of the presentation. What I understand from the mind map that he focuses on the effect of the cloud on behavior and not on the technique. Sounds interesting enough. I will buy his book and tell you later what his story is.

At the end of the day I was invited to an informal dinner with some interesting people. I enjoyed a good dinner and dito conversation. I was back in my room at 23.00 and dived straight into bed. I woke up at 04.00 and wrote this post. The gym will open at 05.30 I will go there and have a good start of day 2.

DevLearn day 0; an energizing day

The pre conference day is almost over. Great day. This morning I registered for the conference and I got to meet some people from the guild that I worked with over the past few months, but never met in real live. And I received the conference  bag with our logo on it, it looks great. After that I had an interview with Bill Brandon. He is the Managing Editor of the Learning Solutions e-Magazine at the eLearning Guild. He will publish his article in the magazine tomorrow. I saw his concept and he does say very nice things about easygenerator. Some quotes are:

“The new version adds content reuse features, workflow management, version control, and great functionality that applies some of the latest and best information we have about learning, in a way that will redefine the usage of learning objectives.”

“The result of this philosophy is a set of innovative features and functionalities, such as the objective dashboard, which sets up the connection between learning objectives and the content of the eLearning. easygenerator separates content from presentation in a way that facilitates re-use and supports localization and translation to other languages.”

You can read the whole article tomorrow in the Learning Solutions e-Magazine.

Next was a meeting with Cheri Bruno with whom I made arrangements for our participation in the Learning Solution conference in Orlando. Then we build up our booth at the expo and after that we went for a quick shopping streak in an outlet followed by some ordinary work (mail and phone calls).

The afternoon ended with a session with Joe Ganci. I showed him the latest version of easygenerator. Joe will review easygenerator this month and we will hopefully improve easygenerator further based on his remarks. I’m looking forward to that.

The day ended with my first meeting with our US colleague Steve Harz, who joined easygenerator last week. Despite a shared jet lag (3 hours for him, 8 for us) we had a great dinner. Steve will introduce himself in a blog on our corporate website later this week. I look forward working with Steve, I have great expectations.

Picture of the day: Steve Harz

Easygenerator is growing rapidly. Next to Steve we had three new people joining us from the first of November. Beatrijs van de Griendt as senior instructional designer, Martin Boon as User Interface designer and Kseniya Kobryn as our new scrum master. I’m proud that I can call them colleagues now. The fact that we can attract great people must mean that we are moving in the right direction.

Tomorrow day 1 of the conference, with two presentations to give, a booth to man, interesting sessions to attend and a lot of interesting people to meet. Now I will review the presentations of tomorrow and put the dots on the I’s and then I hope to have a good night sleep.

Make e-Learning work: Outcome learning (6): the vendors perspective

I started working on the Outcome learning idea to give direction to the product development of easygenerator. And it did generate a whole bunch of new features, but the impact was different than expected. We always considered the e-Learning author and the subject matter expert (SME) as our target audience. But now we added features that had an impact on the learner and for the first time we had to develop features for them. This proved to be a big change. I will try to explain this impact in this post.

We started to develop based on this concept in the beginning of the year. First we created all kind of features that would enable SME’s to play a bigger role in development process. So we created a role based system, workflow facilities and more features like that. To bring learning closer to the workplace we added mobile publications, a QTI export and a facility to reuse content in a smart way. These are all features that were ‘close to home’ for us. But in the version that we are developing at this moment that has changed.

The key functionality of the next release will be learning objectives. We have defined three goals we wanted to achieve:

  1. Support the course design based on learning objectives
  2. Make it possible to assess knowledge and progress based on these objectives
  3. Make it possible to create courses that really adapt to the learner and put the learner in control of his individual learning.

The first two goals led to functionality that affects the author. We created to possibility to define learning objectives, developed a dashboard to support course design based on these objectives, made it possible to connect questions and assessments to the objectives and an option to create a study advice based on the outcome of these assessments. They are all features that we are really proud of, but they are still aimed at the author.

This changed when we started to think on how to bring these new features to the learner. We needed to create an option to present a study advice based on the results of assessments and connected to the learning objectives. The goal was after all to create adaptive courses with true individual learning paths. Easygenerator works with so-called master pages; they define the look and feel of a course and the functionality of the course. And before we realized it we were developing master pages with all kind of extra features for the learner. We created a smart table of contents and a study advice control. We started to realize that our output is not a system to create e-Learning courses, but the courses and the learning experience they offer. Up till now you could not really tell that a course was created with easygenerator or another tool, but now you can. The course itself has unique features and they really impact the way a learner will work with these courses.

From the outside this might seem as a logical step, but for us it is a big change. We are changing from a company that develops software for e-Learning into a ‘learning company’ that develops an authoring platform. Software is not the goal, but it has  become a means to create and change e-Learning. Our complete focus is shifting. As a consequence of this we just hired our first e-Learning consultant and more changes will follow. The funny thing is that this is something I wrote down in our mission statement, but without actually steering in that direction this change just took place. So we as a company have to adept in order to create adaptive learning. This is part of the reasons we chose a chameleon as part of our corporate image, they can adept as no one else.

The only thing might be that we chose the wrong image, it is very possible that it is a Lizard. But we really got used to our green friend, so we will stick with him, being a Chameleon or as a Lizard portraying to be one.

This post is part of a series of post on this subject:

  1. Make e-Learning work: Outcome learning
  2. Make e-Learning work: Outcome learning (2)
  3. Make e-Learning work: Outcome learning (3): The managers perspective
  4. Make e-Learning work: Outcome learning (4): The developers perspective
  5. Make e-Learning work: Outcome learning (5): The learners perspective
  6. Make e-Learning work: Outcome learning (6): The vendors perspective

Make e-Learning work: Outcome learning

Since I became CEO of easygenerator I wanted to have a clear vision and mission on what we do and why we do it. This led to my blogs on our vision and mission, ‘Output learning’ and ‘bringing learning to the workplace’. At easygenerator we use these ideas to give direction to product development and our market approach. Over the past year I joined these ideas, tried to develop them further and I renamed it to ‘outcome learning’.

Although I developed these ideas to give direction to easygenerator I believe that they apply in a broader context. I want to improve these ideas and by sharing them I hope to get some feedback. Therefore I decided to work them out in more detail in a series of post and I will give presentations about this subject at the DevLearn conference in Las Vegas in November and the Online Educa in Berlin in December.

Outcome learning
Outcome learning is an approach for learning in corporate environments. It can help organizations:

– to steer the learning processes differently,
– decide what kind of learning opportunities to offer
– to measure the effectiveness of learning
– make learning smarter
– connect learning to the day-to-day work.
– improve the ROI of learning efforts

It can help the e-Learning developer to:

– make e-Learning of higher quality
– align e-Learning more with business needs
– connect learning to the workplace
– get better insight in the results of e-Learning

It can help the learner:

– to set his own learning path
– shorten the time he spends on learning
– improve on his learning results

Starting points
I based this approach on the following starting points:

– e-Learning is ready to move to the next level.
– we must use learning objectives instead of page progress.
– we need to bring learning back to the workplace
– we need to tune learning more to the learners needs
– e-Learning development is a collaborative process

Three perspectives
The learning process looks very different depending on the view-point you take. Therefore I will describe it from the perspective of the manager, the e-Learning developer and the learner.

This post is part of a series of post on this subject:

  1. Make e-Learning work: Outcome learning (This post)
  2. Make e-Learning work: Outcome learning (2)
  3. Make e-Learning work: Outcome learning (3): The managers perspective
  4. Make e-Learning work: Outcome learning (4): The developers perspective
  5. Make e-Learning work: Outcome learning (5): The learners perspective
  6. Make e-Learning work: Outcome learning (6): The vendors perspective

Get every new post delivered to your Inbox.

Join 1,096 other followers

%d bloggers like this: