Monthly Archives: September 2010

wiki work

Working at Rackspace with OpenStack

I’m finishing up my first partial week as a “Racker” – that’s what employees at Rackspace are called. I’m a special sort of a racker, a stacker, because I’ve joined the OpenStack team as the OpenStack Technical Writer. My wiki apprenticeship and open source documentation experiences with FLOSS Manuals gave me great training for this new role. I’m extremely grateful to get this opportunity.

My first day orientation was great – we got great stories of the startup days, met interesting, passionate Rackers, and toured the facility that is a re-claimed 1.2 million square feet mall (we’re only using 200K of that space in an old Mervyn’s). There’s even the world’s largest word search plastered onto the wall. It’s tough to solve while riding on an escalator.

As I talked to Rackers, I was constantly reminded of my recent read of Tony Hsieh’s book, Delivering Happiness. Apparently it’s on the book club list at Rackspace but I’d say they are already a textbook case study with their fanatical support.

As it turns out, Sarah O’Keefe’s post asking if “all your followers are belong to us” asked some appropriate questions related to my experience, starting as a new employee. While there is no official blogging policy at Rackspace (yep, I asked), I was asked to write my introductory post shortly after accepting the offer, and Content Stacker, Reporting for Duty is the result. Here are the first couple of paragraphs:

Well, hi there. Glad you could make it. Let me introduce myself. I’m Anne Gentle, the newest addition to the hardworking OpenStack team. And I’m so eager to get started I can barely contain myself. I’ve been in lurker mode for a few weeks and now it’s time to reveal my role – I’m the OpenStack Technical Writer.

How did I get here? I first started working in open source a few years ago, and I applied my technical communication skills gladly because there is often a gap in documentation and support in open source projects. I became a big supporter of FLOSS Manuals, where we write free manuals for free software. We work in a wiki and in Booki, a new collaborative authoring platform. We also employ a book sprint technique where we focus collaborative authoring efforts into week-long sprints. I’ve helped with a few book sprints and think it’s a great technique for open documentation.

Where do I go from here? Today I want to start outlining some documentation strategy to get your feedback and tweak it. We’ll be testing doc efforts as we go and likely start with the Object Storage area. Read more

If you think it’s intimidating staring at the blank “page” on your computer screen, imagine my quaking insides this week as I stare at the freshest new wiki I’ve ever faced – wiki.openstack.org. I’ve got a strategy started that I outlined in my introductory post, and executing it with this community is going to be the fun part.

techpubs tools

Web Analytics for Technical Documentation Sites

I’m studying different help sites and applying web analytics. I wanted to write up some of the processes, potential wins, and possible short comings of web analytics for technical communication.

When I spoke with a few Google technical writers at the STC Summit, one of them confirmed that their performance reviews include a web analytics component. This concept got me thinking about help sites I’ve worked on and how well they’d stand that test. Or rather, how well my writing and information architecture would stand up to an investigation with web analytics data. I started looking at what I’d measure. I looked at sites I’ve monitored to find examples. I collected some ideas here.

What are the goals?

I believe tech pubs groups may serve different masters or several masters. Pre-sales or marketing goals are different from support goals, and training or education goals are different still. So you would pay more attention to different measures depending on your goals for the site. This section discusses customer support and preventing costs in the company caused by a support issue being filed. The goal here is preventing support calls.

In the Web Analytics Demystified book, Eric Peterson  points out a distinct difference in goals for a customer support site.

…while the other business models are driven financially by the idea that “more page views are usually better” the customer support model tends to be the opposite, the more quickly the visitor can find the information they are looking for the better.

I’d highly recommend these books for understanding how web analytics tie into business goals. The book has an entire section devoted to Customer Support sites.

For training sites, the goal would be for the person to spend time with the content, digest it, and meet training objectives because they have learned the material fully. Time spent should be higher for training sites.

Another crucial difference between a support site visitor and a training site visitor is that for the support site visitor, you want to observe the behavior of new visitors, but for the training site visitor, you want to ensure retention and repeat visits. To put it simply, customer support deals with acquisition of visitors, training deals with retention of visitors.

Another business goal is conversion – converting site visitors to paying customers. A technical manual can assist with three main goals – acquisition, retention, and conversion.

Support cost deflection – what to measure?

If the site has a question and answer section, compare the page views of the FAQ or Q&A pages to the other pages in the site – are there more views and longer time spent on pages in the FAQ area? That might be a good sign to indicate the help site is deflecting support calls.

Set up a custom segment to look for pages that have troubleshooting or particular error messages in the title or page content. Next, look at the bounce rate for that segment compared to the rest of the site. You want the bounce rate for the troubleshooting topics to be lower than the overall bounce rate. You want the trend for bounce rate for troubleshooting pages to stay the same or go lower over time. In other words, if visitors do not spend any time reading the troubleshooting information you’ve provided, what can you do to improve the content to prevent a visitor from leaving (bouncing)? This screenshot shows an example of a lower bounce rate for the troubleshooting segment of pages compared to all pages, which you want to maintain if your goal is to help users troubleshoot independently.

Pay attention to the pages that have the highest rate of exit, the page that most people leave the site after viewing. But also look at the percentage of exits. I have seen great pages on tech comm sites that have 9% exit rates but the highest number of page views. One explanation is that non-users found them through organic searches and the entire site wasn’t what they expected, not just that page.

What is the responsiveness of people to updates or comments on the site? If it offers comments, how much time elapses between a comment and a corresponding response? Do the questions get answered by a company representative or by another user? Not all web analytics packages will offer this measurement so you may have to do a sampling yourself. You can also try to get a sense of cadence, the rate of comments per day or per week or per month.

Analyzing searches will go a long way towards understanding whether user’s needs are being met by the help site. Look for searches that have zero yield – that is, the user didn’t click through on any hit or no results appeared at all. You can also look at the search results to site exits ratio.

Watch first time visitors‘ data like a hawk. New visitors may struggle at first while they learn their way around the site. As the number of first time visitors goes up, support call volume may also increase if visitors can’t find what they need or if they find the call-in number quickly. A good method for tracking real-world data along with web site data is to use a special phone number displayed only on the online help site, so that you know only those people who found the page with the special phone number can call it.

You can set up a visualization funnel from the home page to the support site to specific information to generate what Eric Peterson calls the “Information Find” conversion rate. For example, consider a flow of visits to the home page, to a product page, to a list of commonly asked questions, to a page containing a specific answer to a question. You can track travel through this series of pages, measure abandonment along the path and track a conversion as a certain amount of time spent on the final answer page.

Potential problems

Limited data will certainly make it harder to provide a convincing, statistically-significant analysis. One of the problems I foresee with applying web analytics on technical documentation sites is the small number of page views per day. I recently analyzed a help site that had about 40-50 page views per day on weekdays, pretty consistently. That was a software-as-a-service product available online. Another site I’ve watched for more than a year consistently gets 200-300 page views on weekdays and just under 100 page views on weekend days. I hear (don’t have an official citation to point to) that the 10k visitors per month is a common benchmark for starting to pay attention to web analytics. Do we get much value or accuracy from analysis if our sites don’t get to that point? I think it’s okay to monitor but to recognize your data may not have the clout you’d like it to.

Benchmarks to compare your site to would be valuable, but there aren’t categories as specific as “help site for consumer gadget” or “help site for enterprise software” yet.

Site search analytics, while most valuable to us, may be harder to enable unless you use specific tools. Site search analysis shows you what users look for, whether they find anything, and the path they take after clicking on a result link. Search analytics focused only on your tech comm or online help site require you to use a Google Custom Search Engine or the MindTouch 2010 platform which has site search analytics built into their reporting system. It appears that Adobe RoboHelp Server Analytics offers the ability to see what users search for but I don’t know the depth of analysis beyond keywords.

Connecting to the greater web analytics group at your company may be a challenge. Google Analytics is the free offering, so I expect it would have the highest uptake in tech comm in the beginning. Also, tech pubs departments aren’t usually tied in to the web content management systems so Omniture or Coremetrics (now owned by IBM) which are two other web metrics tools may not gather data on a tech comm site.

One takeaway from the Web Analytics Demystified book that makes a lot of sense is to always ask, “Is the information actionable?” In other words, when deciding which metrics to watch, make sure you can do something about the resulting metrics, whether you make changes to content or dive more deeply into the metrics. In certain environments, actionable items could be problematic if politics or tools get in the way.

Definitions to Understand

It’s important to get a handle on the basics of web analytics. I appreciate Avanish Kaushik’s blog, Occam’s Razor, for learning about web analytics and the definitions that are so important to understand. What are visitors and what are visits, and what are clicks and what are views? What’s the difference between clicks, visits, visitors, pageviews, and unique pageviews? is a Google Analytics help topic that explains these well. I also pursued and attained a Google Analytics Individual Certification which was extremely valuable for understanding definitions and the mechanics of web analytics.

Sources

I refer to a great article by Rachel Potts titled, What can web analytics do for technical communications? that she wrote for the ISTC’s Communicator magazine last year.

I downloaded and devoured two web analytics books from John Lovett and Eric T. Peterson on their site at www.webanalyticsdemystified.com. The titles are available for a free (give them an email address) download, Web Analytics Demystified and The Big Book of Key Performance Indicators.