I’m studying different help sites and applying web analytics. I wanted to write up some of the processes, potential wins, and possible short comings of web analytics for technical communication.

When I spoke with a few Google technical writers at the STC Summit, one of them confirmed that their performance reviews include a web analytics component. This concept got me thinking about help sites I’ve worked on and how well they’d stand that test. Or rather, how well my writing and information architecture would stand up to an investigation with web analytics data. I started looking at what I’d measure. I looked at sites I’ve monitored to find examples. I collected some ideas here.

What are the goals?

I believe tech pubs groups may serve different masters or several masters. Pre-sales or marketing goals are different from support goals, and training or education goals are different still. So you would pay more attention to different measures depending on your goals for the site. This section discusses customer support and preventing costs in the company caused by a support issue being filed. The goal here is preventing support calls.

In the Web Analytics Demystified book, Eric Peterson  points out a distinct difference in goals for a customer support site.

…while the other business models are driven financially by the idea that “more page views are usually better” the customer support model tends to be the opposite, the more quickly the visitor can find the information they are looking for the better.

I’d highly recommend these books for understanding how web analytics tie into business goals. The book has an entire section devoted to Customer Support sites.

For training sites, the goal would be for the person to spend time with the content, digest it, and meet training objectives because they have learned the material fully. Time spent should be higher for training sites.

Another crucial difference between a support site visitor and a training site visitor is that for the support site visitor, you want to observe the behavior of new visitors, but for the training site visitor, you want to ensure retention and repeat visits. To put it simply, customer support deals with acquisition of visitors, training deals with retention of visitors.

Another business goal is conversion – converting site visitors to paying customers. A technical manual can assist with three main goals – acquisition, retention, and conversion.

Support cost deflection – what to measure?

If the site has a question and answer section, compare the page views of the FAQ or Q&A pages to the other pages in the site – are there more views and longer time spent on pages in the FAQ area? That might be a good sign to indicate the help site is deflecting support calls.

Set up a custom segment to look for pages that have troubleshooting or particular error messages in the title or page content. Next, look at the bounce rate for that segment compared to the rest of the site. You want the bounce rate for the troubleshooting topics to be lower than the overall bounce rate. You want the trend for bounce rate for troubleshooting pages to stay the same or go lower over time. In other words, if visitors do not spend any time reading the troubleshooting information you’ve provided, what can you do to improve the content to prevent a visitor from leaving (bouncing)? This screenshot shows an example of a lower bounce rate for the troubleshooting segment of pages compared to all pages, which you want to maintain if your goal is to help users troubleshoot independently.

Pay attention to the pages that have the highest rate of exit, the page that most people leave the site after viewing. But also look at the percentage of exits. I have seen great pages on tech comm sites that have 9% exit rates but the highest number of page views. One explanation is that non-users found them through organic searches and the entire site wasn’t what they expected, not just that page.

What is the responsiveness of people to updates or comments on the site? If it offers comments, how much time elapses between a comment and a corresponding response? Do the questions get answered by a company representative or by another user? Not all web analytics packages will offer this measurement so you may have to do a sampling yourself. You can also try to get a sense of cadence, the rate of comments per day or per week or per month.

Analyzing searches will go a long way towards understanding whether user’s needs are being met by the help site. Look for searches that have zero yield – that is, the user didn’t click through on any hit or no results appeared at all. You can also look at the search results to site exits ratio.

Watch first time visitors‘ data like a hawk. New visitors may struggle at first while they learn their way around the site. As the number of first time visitors goes up, support call volume may also increase if visitors can’t find what they need or if they find the call-in number quickly. A good method for tracking real-world data along with web site data is to use a special phone number displayed only on the online help site, so that you know only those people who found the page with the special phone number can call it.

You can set up a visualization funnel from the home page to the support site to specific information to generate what Eric Peterson calls the “Information Find” conversion rate. For example, consider a flow of visits to the home page, to a product page, to a list of commonly asked questions, to a page containing a specific answer to a question. You can track travel through this series of pages, measure abandonment along the path and track a conversion as a certain amount of time spent on the final answer page.

Potential problems

Limited data will certainly make it harder to provide a convincing, statistically-significant analysis. One of the problems I foresee with applying web analytics on technical documentation sites is the small number of page views per day. I recently analyzed a help site that had about 40-50 page views per day on weekdays, pretty consistently. That was a software-as-a-service product available online. Another site I’ve watched for more than a year consistently gets 200-300 page views on weekdays and just under 100 page views on weekend days. I hear (don’t have an official citation to point to) that the 10k visitors per month is a common benchmark for starting to pay attention to web analytics. Do we get much value or accuracy from analysis if our sites don’t get to that point? I think it’s okay to monitor but to recognize your data may not have the clout you’d like it to.

Benchmarks to compare your site to would be valuable, but there aren’t categories as specific as “help site for consumer gadget” or “help site for enterprise software” yet.

Site search analytics, while most valuable to us, may be harder to enable unless you use specific tools. Site search analysis shows you what users look for, whether they find anything, and the path they take after clicking on a result link. Search analytics focused only on your tech comm or online help site require you to use a Google Custom Search Engine or the MindTouch 2010 platform which has site search analytics built into their reporting system. It appears that Adobe RoboHelp Server Analytics offers the ability to see what users search for but I don’t know the depth of analysis beyond keywords.

Connecting to the greater web analytics group at your company may be a challenge. Google Analytics is the free offering, so I expect it would have the highest uptake in tech comm in the beginning. Also, tech pubs departments aren’t usually tied in to the web content management systems so Omniture or Coremetrics (now owned by IBM) which are two other web metrics tools may not gather data on a tech comm site.

One takeaway from the Web Analytics Demystified book that makes a lot of sense is to always ask, “Is the information actionable?” In other words, when deciding which metrics to watch, make sure you can do something about the resulting metrics, whether you make changes to content or dive more deeply into the metrics. In certain environments, actionable items could be problematic if politics or tools get in the way.

Definitions to Understand

It’s important to get a handle on the basics of web analytics. I appreciate Avanish Kaushik’s blog, Occam’s Razor, for learning about web analytics and the definitions that are so important to understand. What are visitors and what are visits, and what are clicks and what are views? What’s the difference between clicks, visits, visitors, pageviews, and unique pageviews? is a Google Analytics help topic that explains these well. I also pursued and attained a Google Analytics Individual Certification which was extremely valuable for understanding definitions and the mechanics of web analytics.

Sources

I refer to a great article by Rachel Potts titled, What can web analytics do for technical communications? that she wrote for the ISTC’s Communicator magazine last year.

I downloaded and devoured two web analytics books from John Lovett and Eric T. Peterson on their site at www.webanalyticsdemystified.com. The titles are available for a free (give them an email address) download, Web Analytics Demystified and The Big Book of Key Performance Indicators.

Engaging technical documentation isn’t written by Tina the Brittle Tech Writer. Who is she?

She’s the technical writer in Dilbert’s engineering department. Tina believes any conversation within hearing distance is intended as an insult to her profession and her gender. She strives to maintain her dignity while surrounded by engineers who don’t have a proper respect for her work.

Tina cares more about defensible positions on the engineering department than serving the customer. I love it when I hear people say, “I no longer work for development. I work for the user.” They say it with disruption and evolution in their hearts and minds. They fully intend to serve the user the best they can.

Of course, even the best laid plans can get thrown out the window in a tech writer’s daily work. But here are some ways to engage users with technical documentation. If you’re skeptical that these techniques are effective, go straight to the content analysis of user ratings. Helping 800,000 users in a year is an impressive number.

Go beyond text

People are drawn to images on a computer screen. The Community Roundtable has a great report available that you can glean many content best practices from. For example, the report indicates that “People seldom form relationships with text alone.” Boy, that’s true, and should compel us to incorporate pictures or a video.

I know, I know. Screen captures are a pain to take in the first place, and hard to maintain over time. Let’s think outside the box for a moment. Another way to incorporate images is to use artwork, however simple or stylistic. Take a look at this watercolor created by Oceana Rain Fields, a participant in this summer’s workshop at the Rural Design Collective web site. Illustration used under the CC Attribution 3.0 license (credited to Oceana Rain Fields with permission).

This image sticks with the readers of the manual, especially kids like the child-like figure shown. It should help the users identify with the manual (available online) and get cozy with it.

A friendly, helpful, and confident voice goes a long way in building a relationship in this asynchronous conversation. Screencasting, where you narrate while demonstrating a software feature, is one way to go beyond text in user assistance delivery. As an example, WordPress.tv offers screencasters and users a voice by enabling video uploads to their site. They pre-seeded the site with about 20 professionally-created videos, but after that, users were encouraged to upload videos.

Write informally

Michael Verdi, the content manager for support.mozilla.com (SUMO) has an excellent slide deck talking about Awesome Documentation where he describes some writing style choices they made recently to “Engage the Brain” including inserting humor or surprise and writing conversationally, such as “Can’t decide on just one page? No problem. Firefox lets you set a group of websites as your home page.”   Janet Swisher noted on the FLOSS Manuals discussion list that he also rewrote pages to not just answer what you can do, but why you’d want to do it.

Measure and adjust

Copywriters for campaigns know to do AB split testing – try out two brochures or two web pages on a sampling of people in the database. See which copy and design does best, then stick with that messaging until you see a drop off in use of the information. We don’t employ that technique often in technical communications, but as our copy becomes more web-enabled, I think we should start.

For example, some of the rewrites to pages on the support.mozilla.com (SUMO) site had measurable impact to the helpfulness of the page. For two of the rewritten pages, ratings were enabled. They could measure a 13% increase the number of people who clicked “Yes” for “Was this article helpful?” at the bottom of the How to set the home page article. Because of the high traffic on their site, that’s over 1,000 people per day. Given the number of people who view that page and the similarly-edited Profiles page, the two re-written pages were helpful to 800,000 more people per year. This demonstrates the power of web analytics, especially on high-traffic help sites! This example is fantastic.

Comment and be commented upon

I have many ideas for implementing comments in nearly any online help system in my article on the WritersUA site titled, Putting the User in User Assistance. Comments connect users to each other and to the authors of the content.

Enable storytelling

Your users have stories. Can you find them with some online searches to discover a hurdle they recently cleared or a snafu they’ve found? While case studies are typically under the purview of a marketing department, try to let users tell their stories, or find a way to showcase user stories periodically by linking to their blogs or tweets.

What other ideas do you have for engaging users with documentation? I’d love to hear more ideas.

Last spring I wrote up a blog entry pointing out some additional technical documentation wikis to add to a list I had in my “Wiki-fy Your Doc Set” presentation. A recent Twitter request asking for technical documentation wiki examples brings me back to both lists to try to compile an even longer, more updated list. These are in no particular order and the links were tested in August 2010. Other wikis are behind support logins but this list offers wikis that can be viewed without a login.

• All of the manuals on FLOSS Manuals site are authored and displayed in a customized Twiki wiki.
• Sun Microsystems (now Oracle) has many documentation wikis. The OpenDS Wiki offers a nice example. Also all the OpenOffice documentation is available on a wiki.
• Mozilla Developer Network
• Adobe Labs wiki
• Splunk product documentation
• Atlassian product documentation, specifically the Confluence documentation
• OpenCloud
• GigaSpaces
• Firefox Support Knowledgebase
• Second Life In fact, they single-source their embedded online help with the wiki as source.
• Ubuntu
• RightScale
• ExactTarget
• IBM developerWorks Wiki
• The MSDN Library from Microsoft offers many wiki-like features.
• Embarcadero RAD Studio wiki is actually 9 wikis, read the manager’s blog entry about it.
• WebWorks Documentation Wiki
• Tideway (now BMC Atrium Discovery) documentation

It’s no wonder I have to keep creating new lists. The examples are constantly changing. For example, the Facebook Developer wiki is being moved to another site.

Finally, if you are considering a wiki for technical documentation, I recommend reading my post, Hurdles and Hardships using Wikis for Documentation, reading Sarah Maddox’s blog, buying my book, and sharing your experiences with others. Here’s to enjoying the wiki journey.

I’m pondering the 1998 article, Pages Must Live Forever (from Jakob Nielson’s Alertbox) while documenting the content aging report in MindTouch 2010 (Read the spec here, read the user guide here).

With redirects helping stave off link rot, it seems that we can fulfill the wish behind Kristina Halvorson’s plea not to allow the web become like the junk-filled planet in Wall-E. Instead of piling up old versions of pages, the links stay fresh while the content might age a bit, like a fine wine.

For help content, I can list reasons that older content might be just fine, no need to send off alarms.

• Software that has classic features that were well documented in the first place, those pages can be static.
• Pages that haven’t been updated but are still oft-visited I would consider to be fresh, not stale. As long as the comments don’t indicate a problem with the content, it can be considered fresh.
• Depending on how well it’s resourced or energetic it is, your writing staff and community can only add a finite amount of content per week (or month). So the percentage of old content may be higher than the percentage of new content. That ratio is probably okay as your site ages. The mark the report sets is two years (24 months), then the content might be “old.”
• Depending on the scope of the aging report, an older product would have older help pages. Filtering helps you tune in the grouping of pages where you might be concerned about stale pages.

Two years would be a long time in a web application’s life, but perhaps not so long for an enterprise application. As usual, the answer to “Must Help Pages Live Forever?” is “It depends.” The real question that I’m trying to answer is “When are Help Pages Stale?” I believe two years is a valid and reasonable line to draw. What do you think?

I’m going into a listening phase for a client, where I observe their customers and partners habits online.

I thought I’d write up some of my techniques. The overarching task is listed as “Set up a monitoring system for “listening” to the social media participation by customers or partners.” The deliverable for this monitoring system is in the form of a report. But I’m also thinking of ways to set up a Google account so their technical writers can continue to monitor for months and years to come.

For starters, though, I’m going with who I know as much as what I know, picking the online brains of people who are close to this type of product. First, I searched through the archives of a blog of an industry analyst. I found a great post about a set of opposing videos the company had responded to when they were called out by a larger company (my client is the small company). What a find! Definitely set the tone for what they had recently gone through on the social web.

Next, I set up Google Alerts for the company name and two or three keyword phrases that relate to their products. I found a small group of blogs and bloggers dedicated to discussions about the technology behind the products, but not the product itself. There is also some standards work related to the product, which is good to know.

I also searched on LinkedIn for people’s profiles that have this company’s products listed. Then I made a list of their job titles. This client is already providing me with personas, which is great, but I want to add on more information if needed.

I searched on Indeed.com for jobs in high-tech metro areas where the job description contains some of the key skills the product requires as well as the product name itself. This search also reveals job titles.

I also set up a Twitter search that summarizes keywords and the company name, mimicking the Google Alerts. This search yielded more news and marketing information than I expected, which I could interpret as users aren’t on Twitter, but the companies are on Twitter.

Finally, I want to segment the customers by demographics, such as men in the U.S. aged 35-44, to see what their tendencies are for using online information based on the latest Groundswell Social Technographics ladder and data.

I’m also going to revisit the user assistance research that Scott Deloach put together, assembling best practices for user assistance.

Bug vacuums are a big hit in our house this month. I’m such the mom of boys, huh?

I like the bug vacuums because the bugs aren’t harmed and we can see them up close. Plus, it’s not a terribly loud activity. The bug vacuum hums and the boys just have bouts of mild shouting when they’ve found a cool-looking bug.

A great instruction booklet came with this particular toy. It’s compact, informative, pocket-sized, and I haven’t seen a better quick reference in my life. I immediately thought of Tom Johnson’s nice collection of quick reference layouts and links.

Here’s the first unfold, showing 10 essential field tips.

Unfolding the interior is a real treat – it expands to about eight times the size!

Sure, you could see this as primarily a marketing or sales piece, but only one side of the interior is attempting to sell similar toys – the rest has great instructions and ideas to “Get in the field!”

Anyone else have some “field observations” to share?

I haven’t done a round up of other places I’ve been writing lately, so I thought I’d offer a roundup of articles I’ve written for other sites.

What I’m writing

10 ways to motivate employees to use your CMS – Fierce Content Management

As a content strategist, what motivations help you meet your content goals when integrating a content system? Often the tool selection gets the most attention, yet the motivation of contributors is going to make or break the success of the project. Motivation is a psychological feature–a willingness to act that precedes behavior. You might think of a points system with rewards as a motivation system, but rewards are only one type of motivation. Read more

Putting the User in User Assistance – WritersUA

People on today’s social web are accustomed to participating in conversations, having a voice, giving opinions, offering reviews, and generally interacting with content and with each other like never before on the web. How can we enable users to respond to or contribute to user assistance? The answer could be a wiki, but a wiki is not required to enable more interaction with users. Here are some specific techniques, starting with the simple and moving towards the more complex, including wiki implementation practices. Read more

What I’m reading

I’m also posting reading items to my delicious.com/annegentle account that might interest my blog readers.

First Steps in Flex Screencasts

We meant to do that… (part I) | MsCyra’s Web Development Blog

How developers learn survey results – interesting

WDVL: ‘Users’ Versus People–Understanding What Motivates Online Behavior – Page 2

If there ever could be a Cliff’s Notes for the wiki chapter of my book, I think I’m writing it now. I’ve been working on a great project with MindTouch. I visited them for a focus group with other technical communicators and technical support pros back in February in San Diego. We had open source community documentation represented, we had the health information industry represented, cloud computing, and high tech software writers, Agile writers, and document collaborators. It was a great time, discussing tips, tricks, and the trials of managing lots of content with specific purposes in mind such as learning, education, customer support, technical support, and internal collaboration. The write-up for how to run a focus group of this type is quite good – see Seek Omega: How to hold a Professional Focus Group that Produces Quantifiable Results.

After such a great session, we all continue to talk online thanks to one of the members setting up a LinkedIn Group, and MindTouch also invited us to work on a project to write up specifications for using wikis for technical documentation. We’re basically creating best practices using wikis for documentation, such as:

• templates, such as DITA’s concept/task/reference as well as FAQ and solution guidance through multiple tasks
• tags for workflow, assigning tasks, editing, and categorizing pages
• content collection and curation
• reports that assist with content curation and community documentation

I’ve been circling back with the members of the focus group while I write these specs, and working with Steve Bjorg, the CTO for MindTouch. His attitude towards development is,  create something with a sense of openess and collaborate with users as early as you can. It’s a refreshing way to make software. He describes these first go-rounds as the “Cliff’s Notes” for creating technical documentation with wikis. It’s not as robust as other solutions yet, but it sure does have features that are exciting glimpses at the future of documentation. I’ll post more in the coming weeks and months as we round out the features.

After a Q&A on the Facebook discussion page for my book, Sarah Maddox and I had an additional email exchange talking about the difficulties people face when using wikis for documentation.

I believe that many wikis are in the range of “content management systems” or moving in that direction. But there are many difficulties in general with content management. Here are some areas I’ve heard from fellow technical writers:

Access control: Without being able to say who can view or edit what, some wikis are impossible to apply to tech doc due to serving specific business reasons with the content. A customer support article should not be subjected to multiple edits from wiki spammers.

Hierarchy: Without at least 2 levels of hierarchy, tech writers are stymied as to how to use a wiki without hierarchy. Of course. We have complex documentation sets to maintain and hierarchy is a natural way to organize topics.

Version control: The difficulty in maintaining or tracking several versions of a bunch of topics (or an entire namespace/space) to correlate with a software release version is frustrating to many – I’ve heard this is a basic problem for WordPress’s Codex.

Global search and replace – and don’t forget spell check: Writers are used to maintaining giant Framemaker docs where they could spell check and search and replace across large amounts of content. CMSes and wikis don’t make that so easy as before.

Search on the site itself: We’ve all become so spoiled by Google’s search algorithms that any local search engine usually comes up short.

Workflow: Wikis can be weak in workflow, even as simple as “approve or reject” a particular article.

Creating collections: More than just outputting to PDF, people want to single source from a wiki to create collections of articles based on tags, categories, labels, and so on.

Offline access: Many wikis think they’re the end destination for readers, but the classic scenario is “what do my readers do if they need to get on a plane?” One clever solution to this problem would be to offer the wiki on a USB stick – call it a wikiscicle.

Round tripping: Writers are always talking about roundtripping content. I’ve usually dismissed it as not worth the trouble – there wouldn’t be enough contributions that a team of writers couldn’t keep up with. I’ve finally heard a decent business case for doing so – from structured XML (DITA) contained in a CMS to wiki and back again. Translation (to 22 languages) and volume of edits or contributions are the key to this scenario.

One-click publishing (batch processing): On release day, you want to set all topics to released at once, but with many wikis, you have to go to each page one-at-a-time to click them over to the right state for release.

With plugins and advanced wiki engines, these hurdles are easily overcome. But Mediawiki, a popular wiki engine, flunks the first two tests that many technical writers would apply. These are the examples I’ve seen and some of what Sarah has experienced. How does it match up with your viewpoint?

I’m late to write up my thoughts on Gordon Mclean’s post, Strange Bias, but I give him a belated thumbs up for great self-inspection and data query in the post.

My take? I read ““Why James Chartrand Wears Women’s Underpants” on Copyblogger in December. It’s a great survivor story that you should read in its entirety, but the gist of it is that James is a pen name for a woman freelance writer, who writes the popular blog Men with Pens. Merely representing herself as a man made a real difference in her career trajectory. I was shocked, though, that she never had to talk to clients on the phone and that she never went to conferences or spoke at conferences.

It made me wonder if I’d have 10 times the subscribers to my blog if I had started in 2005 as Tom Gentle. It really did. But we are who we are, and being genuine and transparent is all part of my blogging experience. Many of the opportunities I’ve had in the past 4-5 years are somehow related to my blog and the work ethic it requires to maintain.

And to answer Gordon’s question, “is it just me?” I’d say, my experience with tech pub teams I’ve been on are that men are the slightly minority gender. If you believe Quantcast web stats about the STC website, you see that 61% of site visitors are female. I’ve also observed more women at tech comm conferences than men.

But, socializing being, well, social, means you tend to relate to people like yourself, right? So followers, friends, and fans, being self-selecting as they are, may prove that men follow men and women follow women. I think Twitter certainly reflects this tendency, since research shows men follow men on Twitter. And bloggers use Twitter far more than the general population (See the pie chart on the Day 5 report).

If you read Technorati’s State of the Blogosphere you see that 2/3 rds of all bloggers are men. So the 55% blogs written by men that Gordon reads actually differs from the predictive 66% overall population. A great observation, Gordon, well done.