Questionmark user conference, day 3 #qm10

Time flies when you’re heaving fun. Already the last day of the conference.

There was a lot of fun last night. We did our dinner cruise, that was great. I got to speak to a lot of people, a real networking opportunity. After the cruise I ended up the one bus that didn’t go to the hotel, instead it went to Bongo’s a Cuban nightclub, that was a first for me. I had fun.

Migrating from V4 to V5

The morning started with encore sessions, I attended the session by Tom Gilkison, on migrating from V4 to V5. I didn’t expect to hear a lot of news, but I did. The first part of the presentation was an overview on the differences between V4 and V5, What’s new and what’s gone. Then some do and don’ts on planning and executing the actual migration. The third part was about the new architecture of V5. Tom’s presentation contains all the details. Another document Tim pointed to is the best practice guide, a must read.

Closing Session

The closing session gave an overview of the conference and the conclusions. It was presented by Eric and the product owners (John, Steve, Greg and Jim). Great to see how that concept is working, great team. Some of the highlights for me were:

  • QMlive will be in the end the complete authoring environment.
  • QWmise will become QMwiseR (with webservices)
  • Authoring manager will get a portal like interface using Qframes

All that is left to say is that all presentations can be found on  I’m will take 2 days of now and will enjoy Miami as much as I can. I will be flying home on Friday. It was a great conference. On to  Amsterdam were the European User conference will be in October.

Questionmark user conference Day 2#qm10


I’m getting ready for day 2, I will start with a breakfast meeting with Geoff Woods on some commercial issues. Funny to see that face to face meetings are still necessary in spite of all the social media and communication devices we have. I’m looking forward to the Keynote from David Metcalf on mobile delivery and mash ups. I love the technique, but I’m not sure of the practical value. Then a lot of sessions and the day will end with a dinner cruise. I hate my life.

David Metcalf

A really inspiring presentation on mobile delivery and mash ups. At first I was sceptical, I do see a lot of issues with mobile delivery. But he actually showed solutions that are working and are adding value.  See his handout for details: I bought his book, I will read it in the plane on the way back home. I will probably come back on this in a later blog, interesting guy.


I chose a product central session as the next one, with a very promising title “Innovative integrations with portals Social networks and more”. The conclusion for me was: technically almost every thing goes and for connection Perception to a LMS or portal like Sharepoint we have valid business cases. But what to do with tools like Facebook, Twitter. What is the added value of having assessment there, the concept clearly has yet to grow.

Case studies

I decided to spent the afternoon with Case studies. I’m very interested in hearing and learning from other users, from other countries. It’s always different from the Dutch approach (this says a lot about the Dutch and nothing about the others)

Norwegian assessments

During lunch I talked to two guys from Norway they do a lot of things we (or our customers do to). I think we can learn a lot from each other, I will attended their presentation after lunch. They service nationwide exams for ECDL, Boating and Hunting and in doing so they encounter the same challenges we do. We will definitely keep in contact. See presentation:

Shenandoah University QMlive

A presentation on a successful usage of QM live with their university. I’m a great fan of QMlive and this was a real positive story, so that’s nice to hear, for more details see:

Questionmark Perception as a Central Component of a Training Strategy

A presentation by Citi Financial about how they use perception for E-learning. They actually found a way of using perception as main delivery platform for all e-learning content. They do this because they are not able to use scorm and they want to mix assessments and learning content. It is a very creative way of doing it and it probably works, but I don’t think Perception was created for this task. But you can make up your own mind:

In between session I spoke with a lot of Questionmark staff regulating all sort of things. The coolest by far is that Eric Shepherd will take me Friday to the Everglades, so you see it is not all work. Speaking about fun, this evening a dinner cruise, I hope I have sea-legs. Tomorrow the wrap up.

Questionmark user conference (day one)

March 15

Today is the start of the actual conference. I have a breakfast meeting with Howard Eisenberg. He is one of the leading consultants with Questionmark. We will be discussing a topic raised by one of our clients, after that the conference opening. I’m looking forward to it, the program looks promising to me.

The day started with a breakfast meeting with Howard Eisenburg about a custom solution for syncing data from different applications to Perception, provoking actions like scheduling assessments, deleting users et cetera. Interesting developments both in custom developments and in the standard product.

After that the conference kickoff. Some highlights.

QM Live now supports assessments (new release last Friday). I just did a first quick translation, so it is available in Dutch already.  This means that the basis is complete now. Basic question types, versioning, sharing import and export of question sets and now assessments. Not bad for a years work! Complements to Jim Farell and his team.

Greg was demonstrating new reports that are based on the new dataware solution that will be available later this year,  looks great. I got a change to ask what the relationship is or will be between the data warehouse and RMS and I finaly got it. RMS will be an application that will use the data of the data warehouse. That makes sense to me.

Then I went to a session from Greg on Item and test analysis analytics. Fo r me as a manager important general development.

After that I  went to see a presentation from Eric Shepherd on Remote Video Monitoring of Test Takers. Two companies developed intriguing machines and software to monitor on a distance people undergoing assessments. Very interesting, but I’m still processing this and trying to figure out what the possibilities are in the Netherlands. At the end of the session I talked to Jim farell about the secure browser. Jim will provide me with dates on which the new version will be available.

Next I had a meeting with Brian McNamarra (Marketing Director Questionmark) on the European user conference that will be held in Amsterdam on 4  and 5 October. So if you live nearer to Amsterdam than Miami, clear up your schedule and book this. I’m very much looking forward to this event. We put all the dots on the i and now we are ready to go.  We also talked on the new community program Questionmark is implementing, a big improvement over the old facilities, it will be live through shortly. Make a note!
Because the meeting took longer than expected I was late for the benchmarking session of John Kleeman and Greg Pope. But I got the essence. Comparing your results to other similar instituations/companies could give great inside. This is base on very interesting stuff.
Finally I was able to attend the last half hour of a workshop over scorm and standards, let by Steve lay and Daniel Rehak of ADL. Again interesting but complex stuff. Now I’m reading my mail and writing this blog. I’m invited by Eric Shepherd for dinner with an interesting bunch of people, sounds promising.
Tomorrow morning starts with a breakfast appointment with Geoff Wood (VP sales Questuionmark) on some commercial issues.

Questionmark user conference

I’m attending the QM user conference in Miami, and I will try to log the events in a series of blogs.

March 13

I arrived in Miami on Saturday evening. Quite a bumpy trip. I had to make a stopover in Philadelphia, but the weather was so bad we got redirected to Banger (wherever that may be). In the end we did fly direct to Philadelphia, but it wasn’t a smooth landing at all. My connecting flight was delayed so I arrived at the hotel at 23.00. That’s 20 hours door to door.

March 14

After a few hours of sleep I got up early and wandered around in the downtown area of Miami. I walked up to the bay side market place and had a nice breakfast at a harbor.  I’m not impressed with the downtown area. Back in the hotel I joined a QM conference group, we where supposed  to go on a go-tour ( gps guided tour) but something went wrong in de reservation. We ended up doing the duck tours, that was fun. You step into an amphibious vehicle, drive around and then plunge into the water. it was fun. After the tour I started discovering Miami beach, that’s the place to be. Miami is a bit strange. Downtown Miami is on the coast, Miami beach is an isle 2,5 miles in to ocean, they are connected through a series of bridges, that run along a number of small islands. All these islands are artificial. They are home to the rich and famous. Miami beach is the place where it all happens, I walked around for a few hours enjoying the city. I think I don’t mind that I have a little extra time after the conference.

At 19.30 we had the informal opening of the conference (a pool side reception). Got to meet a lot of old acquaintances and made some new. Made a lot of appointments with Questionmark staff for the coming days. After the reception we went into town for a dinner. Raphael (who lives in Miami and is Argentinian) took us surprisingly to an Argentinian steakhouse, it was great. It was close to one before I was back in my room. Tomorrow the real work starts.

QMlive: The way software should be

One of the roles I have at Stoas is the role of Product manager for Questionmark Perception.  I have that role since I started working for Stoas, three years ago. John Kleeman started Questionmark some 20 years ago, building the first version at his kitchen table. John by the way is one of the most special people I know and someone I greatly admire. Over the past twenty years Perception grew out into the world leading software for assessments. I don’t know the exact numbers but they are quite impressive. Stoas is representing Questionmark in the Netherlands for more than 10 years and we have over a 100 customers using Perception intensively. It is our most important product, both in number of customers, sold services and overall revenues.

Questionmark is a ‘standard’ software company building proprietary software. We had version 4.2 in 2007, 4.3 in 2008 and version 5 in 2010. The software is improving with each version. Version 5 for example is a complete rebuild of the delivery part of the software (their crown jewel) but …. it took them three years. I understand that, you have all kind of legacy systems, installed bases, tons of software code to rewrite et cetera. But it is the old kind of software development. Let’s say it isn’t an agile approach. But Questionmark is learning and changing. An example of that is their new service for Subject Matter Experts (SME). It is called Questionmark Live! and it’s a light webbased tool to develop questions, aiming at subject matter experts and I became a fan instantly.

QMlive authoring process

Why is it so special?

Questionmark appointed a new guy (Jim Farrel) and made him responsible for QMlive  and gave him a small team of developers. They created rapidly the first version of  a simple authoring tool and put it live in April 2009 as a service for all Questionmark customers. Unlimited and for free! As a good cheap Dutchman that’s the first thing I like. If you have a support contract on the Perception software, you can use it. Even if you just have 5 licences for authors, you still have access for an unlimited number of QMlive authors.

It’s great software. Jim and his team are building beautiful software. The facilities they have created for versioning and sharing are excellent, exactly what I wanted them to be, and maybe even more important it has a really great and intuitive interface. Anybody can create and share question with no training at all!

It’s fast and agile. Jim and his team are constantly updating and expanding the software. On average there is an update every two weeks. They can do that because all the software is on Questionmark servers. Not only are they adding new features but they are listening very closely to the user demands and giving them what they ask for.

It’s multilingual. They created a facility to translate the tool online. I am the volunteer for the Dutch translation. If they put a new version live on Friday, I will translate that on Saturday and it will be instantly available in Dutch. Of course sometimes I mistype a word or make an other mistake. But then I will get an email from a customer and  I can change it on the fly. I like this way of working.

It’s very successful. Our customers like it. We started in April 2009 with no users at all, in September we had 560 unique Dutch visitors creating 5000 questions. In January we had over 1500 unique Dutch visitors, creating over 13.000 questions.  This is an average of 15 users per customer! Incredible numbers and great fun that you can monitor the success in this way. By the way, this makes the Netherlands the second largest user of this service worldwide, only the preceded by the United States. That makes me proud as a Dutch fan.

It is the future. Software like this and the way it is created and offered will be the future. I complement Eric Shepherd and his team for having and following this vision. It will ensure that Stoas will still be a proud Questionmark partner in ten years time.

Curious? You can test this software yourselve, ask for a demo account. Just click here for more info.


Get every new post delivered to your Inbox.

Join 1,101 other followers

%d bloggers like this: