During a CMS selection project someone said that he thought the CMS systems he had seen demonstrated appeared to be the same! The reasons for this were:

  • They performed the same functions e.g. content publishing, versioning, workflow, searching etc.
  • They used similar ter`minology to refer to the products e.g. Content Server, Deliver server, modules etc.
  • The vendors presentations were performed in much the same way, e.g. complany profile, client list, product demos

I agree, that CMS demos can be a bit of  a blur especially when you have a few on the same day! Most clients want to focus on content, usability, and their audiences, and they do not want a CMS selection project to be about “technology”. CMS selection teams should consist of staff from different departments and skillsets e.g. IT, finance, Marketing, Press and Communication, and it is easy for them to become overwhelmed with the amount of information they need to make sense of.

With an estimated 4000 CMS systems in the world there is a vast amount of choice out there. CMSWatch covers around 40 products as part of their WCM report, so the evidence suggests that they are not the same! Even on a shortlist of 2 vendors they usually differ in their :

  • strengths and weaknesses
  • licenses models,
  • costs (and discounts),
  • implementation partners,
  • customers,
  • sizes and types of projects,
  • architecture etc.

So next time anyone in your team says “but they are all the same!” whack them over the head with this post!

See Also

5 Biggest Mistakes in CMS Selection

In this article I’m hoping to present some of the options available to you to assess the usability of a CMS you have on your shortlist. This article discusses steps you can take to help you build a picture of how the CMS functions and how it might be used to fulfil your needs.

I thought about writing this article because :

  1. CMS usability is very difficult for vendors to attain, most CMS are very complex consisting of suite of applications catering for users of all abilities (from junior editors to system administrators).
  2. CMS usability is difficult to assess. You may not have the time or resources to give the product a proper test drive, or be overwhelmed by the sheer amount of functions available.

Developing use case scenarios

Stephen Krug’s book “Don’t make me think” tells you precisely how you should go about usability testing. You’ll need a small number of authors and editors (at least 3-4 to make it worthwhile) and a list of scenarios which encompass the most common tasks that you will be performing, these scenarios might include:

  1. Creating new content. Creating a new article (with images) and promoting it on the home page, relating articles – either directly or via meta data (such as a taxonomy)
  2. Updating and approving existing content (include user generated content). This would demonstrate workflow and approval processes and versioning capabilites.
  3. Updating design. Modifying layout of templates and page design elements

Getting your hands dirty

There are a number of options presented below, each depends on the time, budget and resources you have at your displosal. You are likely to be looking at more than one CMS during the vendor and online demos, and looking at going on training and a procuring a proof of concept only for your preferred solution.

  1. Vendor demo
  2. You go through the product with the vendor, a well-managed vendor demonstration should provide a pretty good idea about the usability of the product. These demos usally last around two hours which is usually not enough to get an in-depth understanding of the product, but it’s a good start, and should help you narrow the field.

  3. Online demo
  4. Most vendors will be able to provide you with an online demo site for their CMS. This offers a simple way of playing with the product in your time and using your own equipment. The demo won’t be populated with your content or have your site structure, however it could give you a good feel for how everything fit’s together. A word of warning though as almost every CMS requires training, simply playing with a product could be potentially misleading and can become very easily frustrating!

    Ideally you would want to run through your most common scenarios on the demo without too much input from the vendor. If you can without getting confused or disoriented then it bodes well for how others will be receptive to it.

  5. CMS training
  6. Ask the vendor about end user training for the CMS. This will help to answer many questions: is training available? Some vendors don’t provide user training as they believe that their product doesn’t need it! If it’s needed how long is it? 1 day or 5 days? Some authors could be sent on training to find out how simple the product was to use? Can the authors understand the product at the end of training?

  7. Proof of concept
  8. You should ask the vendor to create and deploy a proof of concept based on your scenarios for you to work on. You can then conduct scripted tests with trained users to help identify any major issues.

Even with using the techniques presented here, CMS usability can still be difficult to assess e.g. what is simple for one content editor could be difficult for another. Nevertheless, I believe it is vital to conduct an evaluation of usability as part of the CMS selection process, and it could be the difference between project success and failure.

Is usability testing a part of your CMS selection process? What has worked for you?


Don’t make me think (usability testing by Stephen Krug)

Practical ways to assess CMS Usability

When speaking to clients and CMS vendors we have been discussing what is the most important criteria that CMS buyers are looking for when selecting a CMS. A year ot two ago I would have said it was cost or how well a CMS met requirements. However in our most recent projects usability has been as important as cost or how well the CMS met a clients’ requirements. So why is it important and why does it feature at all in CMS selection projects now and what do we mean by CMS usability?.

What is CMS usability anyway?

Usability means making sure something works well, and that a person of average ability or experience can use it for its intended purpose without getting hopelessly frustrated. Therefore a usable CMS will :

  1. Be easy to learn
  2. Be easy to remember (particularly for occaisional users)
  3. Have a simple content management model (content types, ‘pages’) and follow a users’ mental model
  4. Require minimal training and be easy to explain to non web savvy editors.
  5. Be fast, resilient and error proof.
  6. Provide helpful error messages and tips when things go wrong. (wishful thinking!)

Clearly these things are not always covered in a statement of requirements, and have a significant impact on a successful CMS project.

Why is usability so important?

  1. Project failure is not an option (particularly now). If a CMS has been selected based on primarily technical or value for money criteria then there is a siginificant additional risk the project will fail!
  2. If  your team does not understand the CMS they will not use it.
  3. You will have to factor in additional hidden costs of training (of existing and new staff) and support. Your team will be inundated with support requests from users who will get stuck and then your team will spend time helping them.
  4. Devolved content management will not be possible, without significant additional investment.

What do you think? Does usability feature in your CMS selection? If not why not?

Useful posts on CMS usability:

  1. 11 usability principles for CMS products (Step Two Designs)
  2. The 5 hidden costs of running a CMS (Paul Boag)