Twitter for usability testing or doc testing? Sure, here’s how

My coworker went to SXSW Interactive this year, and I merely went to lunch with people near SXSWi and followed the #sxswi Twitter stream, and now I am browsing through the sketchnotes. I went to BarCamp Austin on Saturday and presented about FLOSS Manuals and the Book Sprint methodology we’ve been experimenting with. It went great, and afterward I even got a nice compliment “You are a force of nature, aren’t you?” That made me grin big!

But back to Twitter for usability testing. How can that be done? The CMS Showdown featured at SXSW Interactive actually came up with a way to do it, and then there’s a video on Vimeo showing one of the participants watching and commenting as the Twitter stream goes by on her screen.

JoomlaSxSW AmyStephen’s Review of the Review from Amy Stephen on Vimeo.

Basically, set up a time for a certain number of website users to try certain tasks on your website or application. While they use the application, have them log on to Twitter and make comments, including a pre-set hashtag in their Tweets. By the end of the testing period, you’ll have a record of micro-comments (140 characters or less) collected with the search.twitter.com tool.

Somehow this use of Twitter to “judge” your product or documentation makes a lot of sense to me. You pick a hashtag and a span of time, and ask people on Twitter to read the doc or try the product at the same time, putting their thoughts up as 140-character or less Twitter posts.

Now, be sure to save off the stream of comments because, as Jenny Levine noted, the stream of the moment is momentary.

A week or so ago, people in the technical education sector did something similar to what I’m suggesting – they all discussed a topic at the same by putting the hashtag #educhat into their Tweets. We’ve been talking about a similar organized chat time for FLOSS Manuals.

I’ve blogged about uses for Twitter before – usability testing is just one more use to add  to the list.

Subscribe to the comments for this post

Examples of blogs as online help and release notes

I’m always on the lookout for examples of social media tools used to write and maintain online help. One trend I think I am seeing is the use of blogs as the basic release notes for new features in products, especially web applications. Examples are new Google Calendar features and SmugMug, where the entire blog is dedicated to Release Notes.

I’ve also found the Jing online help is written and maintained in Movable Type, a blogging tool. Many blogging tools can be used as content management systems, and it appears that Jing’s writers see blog engines that way too. There are lots of nice built-in features that they are taking advantage of – a nice Search field at the top of every page, and the Categories link at the bottom of each help topic give a nice collection of topics. There’s only one “table of contents” for the help system, and that’s the top page, but it works nicely as a site map. The overall effect is a very simple and elegant user assistance or support system. One detail I did discover while trying out the site, though, is that the MT search engine did not find hits for a search on “mpeg 4″ when the topic titled contained MPEG-4.

The use of a blog overall seems like a great idea for release notes – give your product some Google juice and search power as well as generate buzz for new features by giving other bloggers a well-understood infrastructure to link to you and give your entries trackbacks. If your release notes contain a lot of bug reporting or issue fixes, I’m not sure a blog is a good match since that’s not exactly a positive spin on your product release. Then again, sometimes transparency and honesty is the best policy. What do you all think?

Subscribe to the comments for this post

How many roles can a documentation expert have in a company?

Harjot Dhodi asked, “What is the difference in the role of: Document Architect, Template Designer, Writer, Technical Editor, and Production Editor? Can a person be told to handle all these roles?” I’ll try to examine these roles one at a time, and then answer the final question last.

Document Architect – Typically this person has a “big picture” view of the documentation and how to organize it to fit the user’s needs. This role involves organizing, dictating what topics will be written, structuring the overall deliverables (especially if there are multiple deliverables such as online help and printed documentation). I would say this role is for a more experienced person who has been with the company a while and knows the business needs for the documentation.

Template Designer – This person knows the documentation tools well enough so that they can maintain and design templates used over and over for consistent documentation while authoring so that the out put look and feel is the same over and over. For example, the role would involve designing FrameMaker templates for books and chapters and styles for the authors to use while writing.

Writer - This person creates the content. They should be familiar with using the templates and the style guide for the company.

Technical Editor – This person reviews the content and may also maintain the style guide. Some times the term “technical” in an editor role means they will check the documents for consistency with the product and technical accuracy throughout the document. Grammar and style rules checking is also part of the responsibilities of this role.

Production Editor – This phrase is less familiar to me. I would imagine that this person reviews output for any errors and does link checking for online deliverables. I suppose it really depends on what deliverables are produced. The Production Editor may need to check CDs to ensure the documentation deliverable operates correctly on the CD. There could be a lot of editing and testing and checking on certain production deliverables.

Can a person be told to handle all these roles ?

I think one experienced person could handle all these roles, and a single person could learn one role at a time and just keep adding each role to their abilities. In my company we do not have separate editors so we must review each other’s content. We all write content, we all edit content, and we have a document architect and production editor who is most familiar with the content and production tool.

Does this help answer questions about documentation roles? I’d love to hear feedback from my blog readers as well – what have I missed or what additional roles might there be placed upon one person?

Subscribe to the comments for this post

Upcoming presentations in Austin and Houston

I’ll be presenting Documentation as Conversation at the Austin STC meeting Tuesday October 7th.

Austin When

Tuesday October 7th
6:00 – 6:30 PM: Networking
6:30 – 7:30 PM: Program
7:45 – 9:00 PM: Networking dinner

Austin Where

UT Commons Center – Room 1.138 [Map]

Then in November I’m going to make the trip to the Houston STC Meeting Tuesday November 11th.

Documentation as Conversation: Working conversation and community into documentation using social technologies

Even if your documentation system does not converse with your users, your documentation can help customers talk to each other and make the connections that help them do their jobs well, play with technology at home, or learn something new in a classroom setting.

Instead of concentrating on single sourcing or the tools of the trade, this talk describes how you can think about documentation and user assistance in a conversational way, perhaps with the help of some social networking applications. I’ll also discuss the in-person FLOSS Manuals BookSprint as a use of a wiki paired with a community event to gather together writers to accomplish documentation goals for free, open source software projects.

Subscribe to the comments for this post

DocTrain West 2008 – Bob Glushko, Document Engineering

Bob Glushko blogs at docordie.blogspot.com, great blog name and a fascinating presentation. I liked that he shared and described his semi-retirement as verbalizing his desire to be a beach bum to his wife, but his wife said, I still like my job and I want to work, so go get a job! He has been teaching at UC Berkley ever since.

Building information supply chains – example of the E. Coli scare in lettuce in March 2007. Basically had to figure out how to track heads of lettuce, similar to tracking heads of people to avoid long lines at security in the airport. With enough data tracking – input and retrievability – you can make informed decisions.

Common themes of new information services – document exchange, patterns, similar to supply chains and distribution channels. There are hidden documents in business processes.

His “ah-ha” moment? he had always focused on the document, but with ordering on the web, his user experience is what really matters – did the business process work? Did the lobsters arrive dead or alive? Did his shipment get to him in time and was it the right order? You have to know the back-end, the time difference, the travel distance, the choreography and design of the pattern determines success and a happy user experience.

I’m reminded of the fact that there are 39 time zones in the world, and for collaboration across the world, we have to figure out the time zone difference relative to the person you want to collaborate with.

Bob offers an excellent analogy for wiki-based, community-collaborative content – a restaurant’s lines of visibility. At McDonalds, you have backstage production lines for food prep, at Benihana you have food prep as part of the entertainment right at your table (remeber that onion volcano so expertly prepared?) We should try to strategically determine where to draw our lines of visibility – what point of view do we wish to present to our users?

Ah, now he’s talking about a cooking school where the kitchen is the front stage for the cooks, and the back stage for the customers. A restaurant’s dining room is the front stage for the customers, but the back stage for the cooks. I’m reminded of a webpage I read where people proved that writing on a wiki actually helps you learn more about the tasks because you have to figure out your conceptual understanding of the task to write about it. If you allow more writing to happen next to the backstage when it’s the cooks in the kitchen, or the expert writers in the wiki, more beginners can learn by not observing or reading but by actually participating in the writing itself.

While you may have identified more with either the front end or back end design issues, you can choreograph the information experience for the user.
Here are Bob’s slides, also found on slideshare.net.
[slideshare id=384924&doc=doctrainglushko-1209742956786846-8&w=425]

Subscribe to the comments for this post

Amazing conversations and meeting amazing people at SXSW Interactive

South by Southwest Interactive is a big web design/blogging conference in downtown Austin. There are thousands of people here for it in 2008. It’s crazy busy.

The latest excitement was in a friend’s panel where people tried to stage an uprising on both Twitter and in the meebo room at meebo.com/sxsw, set up specifically for this talk. Tom Parish has a great picture and is quoted in this Wired blog story. The very next day, the Zuckerburg interview had a similar uprising on a much grander scale. Jeff Jarvis has the best analyses I’ve read so far about it, and Scoble’s view from the Twitter gallery is also a good read.

I’m of two minds about the things that have happened here this week – on one hand, I think the conference is only as high quality as its presenters. If all the attendees think they’re smarter than and better than the panelists, then why bother coming at all when you can view the video online or listen to the podcast later? I guess that hypothetical question is answered with – we come because we can interact with the panelists.

I even witnessed a panelist admit that he “wasn’t paying attention” to another panelist’s answer to a question during their panel. It came across as immature, arrogant, and unprofessional to me. Much like the sweater-tossing antics I observed based on the meebo room conversation in the social media metrics talk, I internally rolled my eyes and thought, how many people are just trying to get attention, drawing it away from the panelists disrespectfully? Is this online behavior and real-life behavior only as mature as the junior high lunch room?

On the other hand, if all the panelists preach about user-centered content, then when the choir stages an uprising, the preacher should be able to adjust his message to fit the audience. Right? I really admired Tom’s quick thinking. I was admiring the way that Tom handled the panel, ensuring that the podcast recording turns out well also, by having each speaker introduce themselves so that listeners later can identify voices while listening.

But enough about the conflicts and struggles going on between panelists, interviewers, and SXSWi attendees. This year, my interaction with other attendees has been the most exciting and fun for me.

Before and after the social media metrics talk, I befriended two guys from Washington, DC, and Summer from Austin who were all sitting around me.

I ended up giving a ride up Sixth Street to David, one of the guys from DC, and he and I had the nicest conversation on the way to BarCamp. I had asked about the conversations that occur on wikis and how interesting it was that the WoWiki panelist George Pribel said they never want to answer how to or troubleshooting questions on their wiki, that they only wanted articles and discussion around strategy. I said that as a tech writer, I was looking for the best use of wikis for content, but since we usually live nearest to and offer the most value to the customer support department, our wikis would be shaped more towards howto and troubleshooting information. However, best practices and strategy wikis might be more easily shaped for conversational articles, so, which was the better approach? His answer was spot on and an excellent example that will stick with me. He said, it’s just like the real world conversations you have in a crowd. If you and I are talking about movies, and someone comes up and starts talking about their favorite restaurant, we would politely inform them that we’re talking about movies, and could you take your food conversation elsewhere, maybe to the person sitting next to me? It’s a matter of staying on topic. With wiki design, I would conclude, you might want to prepare for two audiences (and two types of conversations), just like Lisa Dyer has done at Lombardi Software.

After attending David’s talk, WhiteHouse 2.0 at Barcamp, I learned that he’s former White House Internet and Communications Director David Almacy! He started RSS feeds and podcasts on iTunes and Tivo for the president’s white house. He wrote a Barney Cam script that got posted on Youtube and had over 25,000 views that season. As it turns out, he has two daughters nearly exactly the same age as my two sons. Our youngest kids were born within three days of each other. He was so nice and professional, a knowledgeable expert who is also willing to share his experiences. Prior to the social media metrics talk, I babbled about how I was a blogger for Ynema and Tom on talk.bmc. Little did I know of my fellow attendees level of experience with social media, but I was so pleased that David didn’t try to prove just how much more knowledgeable (and famous) he is than I. Instead he just answered my questions and truly listened to me.

I’m still amazed at the serendipitous meetings and conversations. Yes, the attendees make the conference interesting, and the panelists are bravely facing those attendees.

Subscribe to the comments for this post

Wiki for documentation

Stewart Mader has created a series of videos available at www.ikiw.org/21days called 21 Days of Wiki Adoption.

Each video is short, encapsulated, and easily digested when you need a break. I’m really enjoying them, and the cool US map background behind Stewart.

Day 12 is Documentation. Great ideas there, involving authoring in the wiki and using the wiki engine to publish a PDF. I’m working on a blog entry describing authoring in DITA and outputting to a wiki, but it’s also nice to hear of the other way around.

Subscribe to the comments for this post

The Rockley Blog – wiki’s delivery mechanisms

I’m thoroughly enjoying the new Rockley Blog at http://rockley.com/blog/. I’m so glad Steve Manning and Ann Rockley are blogging, especially about wikis.

I appreciated Steve’s post “Wikis for Documentation” especially where he says that the delivery side is the weak point still. Agreed, but I have seen pockets of improvement there and need to be blogging heartily about them.

About three years ago I was equally as unconvinced of a wikis usefulness for end-user doc. An Agile-advocating developer mentioned the idea of using wikis so that the end-user doc could stay in sync with their fast Agile iterations. Yipes I thought! Wikis did work well for convincing the developers to contribute their knowledge, though, and internally they became a useful knowledge sharing (and finding) system.

Now I’m starting to see more and more actual customer need for them. I just had a great discussion about the difference between what a customer needs and what a customer thinks he or she wants, though, so some of what’s necessary is interpreting whether a wiki can fulfill a customer need. I got a small chuckle out of the title of this blog entry – Wikify Documentum Already – but he’s talking precisely about the gains you and your customers make when documentation is in a wiki. The interesting momentum going now is whether the current large enterprise content management systems can start to see the value in a wiki output, or whether the wiki engine providers themselves are going to catch up with the full feature set available in the large enterprise systems.

But wow, I’m learning how difficult wiki maintenance and trust patterns can be while using the wiki.laptop.org site to help out with their end-user doc. In some ways, wiki doc is more difficult than using the real CMS tools that we’ve become accustomed to (read: spoiled by). But I’m also learning how amazing the collaboration opportunities are when using a wiki. I’m still marveling at the communication going on in the discussion pages as well as the volunteer spirit that has come through a request on an art network.

So, what to do to get decent output for content delivery using the multiple channels that us advanced single-sourcers are already accustomed to? I’m planning to move the XO’s end-user doc towards the Flossmanuals.net model of a highly customized Twiki implementation where you can get and print a PDF from the wiki. I can’t wait to learn more and I’ll blog about it as I find out. It’s a step in that direction, though, where you deliver user guides and online help and web sites tailored to the needs of specific audiences. Language translations in a highly distributed environment are going to be an important part of the project, and I’m curious about how Flossmanuals provides for that aspect.

I’ve learned that you can write a Confluence plug-in that will take DITA source and turn it into wiki text. Confluence has PDF output capability as well, so it’s another step in the right direction to get that just-in-time content delivery that a customer needs (but doesn’t know that they want.)

Putting documentation in a wiki (or any really-well-indexed web location, really) can increase findability. If you get internal comments that say “you haven’t documented this particular feature enough” and you feel the feature is sufficiently documented, examine the findability of your documentation.

Also in our user advocacy role we are learning how to listen to customers and then interpret their needs. As information acquisition continues to gather speed, we not only provide the information but should also make informed choices about delivery methods.

Examples of what a customer wants but might not know that a wiki can deliver:

• What’s new with the product?
• How do I interact with documentation, support, the company represented as an actual live person?
• I want immediate information updates.
• I want to discuss the nuances of an implementation decision.
• I need to find others who are attempting what I am in the same type of field (insurance or banking).

What are your thoughts? Are we spoiled by our advanced delivery systems and waiting for wiki engines to catch up? Or are wiki authoring and delivery systems already giving us collaborative opportunities that are unparalleled?

Subscribe to the comments for this post

What makes a doc plan Agile?

Does working in an Agile development environment change the documentation plan, or user information plan?

For many writers, the purpose of a doc plan is to inform the stakeholders what you (or your team) intends to deliver for docs for a particular product release and when you will meet key milestones.

Here’s an outline of a typical user information plan that includes signoffs from interested parties, with the goal being that you work with your team to agree to what information deliverables are necessary for a software release to be complete.

1. List of deliverables
2. Risks and dependencies
3. New features
4. Documentation milestones and deadlines
5. Techpubs expectations and assumptions for meeting the deadlines

That outline was from an actual doc plan from about three years ago. It was feature-dependent and waterfall-driven.

In an Agile environment, the doc plan might be more minimalist, listing only:

1. key players
2. release theme and top features in the release
3. dates

Then the real work of planning the details of what needs to be addressed in the doc happens during iteration planning. The detailed planning may not be written down in a doc plan, spending the time on writing the end-user doc instead.

I suppose the most Agile doc plan would be a simple verbal agreement to what will be in the doc each iteration.

If there’s a sliding scale, what is the least Agile doc plan? What is the most Agile doc plan?

Subscribe to the comments for this post

Making the documentation cruft calculation more user-friendly

Our tech pubs group enjoyed Scott W. Ambler’s “Calculating Documentation Cruft” article in Dr. Dobb’s Journal related to Agile methods and documentation. We had some fun in an email thread and I’d like to capture some of our discussion here.

Scott cleverly turns CRUFT (a geek term that either combines crust with fluff or refers to a lab where useless objects piled up in the hallways) into a measuring stick for the effectiveness of documentation:

• Correct = The percentage of the document that is currently “correct”.
• Read = The chance that the document will be read by the intended audience.
• Understood = The percentage of the document that is actually understood by the intended audience.
• Followed = The chance that the material contained in document will be followed.
• Trusted = The chance that the document will be trusted.

Typically, I focus on Agile and end-user documentation, but Scott dwells on the type of documentation that you use while developing the software. Knowing that the Agile manifesto doesn’t value comprehensive documentation over working software, the fact that he’s calculating cruft in a document is refreshing and unexpected. However, my co-worker Shannon Greywalker and I both thought his algorithm could use some refinement. Shannon, being a math whiz, was able to re-mix it and gave me permission to post the idea here.

Shannon says “His formula is essentially 1 – (C * R * U * F * T) = a flat percentage “cruft” rating, wherein higher values are worse.

Let’s assume a perfect score for all 5 values (100% or 1.0 for each). It’s 100% correct; will have a 100% chance of being read; and will be 100% used, followed, and trusted.

1 – ( 1 * 1 * 1 * 1 * 1) = 0.0 = 0% CRUFT. An ultimate low score and so far so good.

Now let’s assume a 99% score for all 5 values:

1 – (.99 * .99 * .99 * .99 * .99) = 0.049 = 4.9% CRUFT. Our score has jumped almost +5% from 0. Still looks okay, mostly…

But now let’s assume scores of 40%, and 30%, for all 5 values:

1 – (.4 * .4 * .4 * .4 * .4) = 0.989 = 98.9% CRUFT.

1 – ( .3 * .3 * .3 * .3 * .3) = 0.997 = 99.7% CRUFT.

The curve produced by his formula is sharply logarithmic instead of linear. Logarithmic curves are the hardest for most of us to use in everyday life just because most of us can’t natively compare one logarithmic value to another.

The best example of an original formula that is not human-intuitive and therefore not all that useful is the Richter scale for earthquakes. The average layperson has no idea what those numbers really mean because 6.0 doesn’t sound that much worse than 5.0 – but it is much worse, because the Richter’s a logarithmic scale.

A more straightforward, intuitive, and linear results curve would be expressed by the formula:

1 – ((C + R + U + F + T ) / 5)

Examples:

With a 60% score in all 5 factors: 1 – ((.6 + .6 + .6 + .6 + .6) / 5) = 0.4 = 40% CRUFT
With an 80% score in all 5 factors: 1 – ((.8 + .8 + .8 + .8 + .8) / 5) = 0.2 = 20% CRUFT
With the scores from Scott’s example: 1 – ((.65 + .9 + .8 + 1.0 + .9) / 5) = 0.15 = 15% CRUFT ”

I would take it one step further, with these descriptive ratings:

40% CRUFT, “document is getting a little crufty” rating

20% CRUFT, “the cruft hasn’t yet overtaken your doc” rating

15% CRUFT, “this document is approaching cruft-free” rating

Shannon’s argument is a good one, the best way to increase the understandability of the CRUFT rating for documentation would be to create a linear calculation for it.

Shannon also argues that there should be a weighting factor for each of the five factors used in scoring a cruft rating. Now, I say that people would disagree on which factors are the most valuable in a document, so weighting might overly complicate the calculation.

I also find the weighting to be too subjective because it seems that Correct could influence Trusted (docs that are found to be incorrect are then distrusted). Also, Understood is elusive for a writer to measure and also to target that specifically – if a selected audience member doesn’t understand what you’ve written, is that always the fault of the writer?

I do like Scott’s tips for creating less crufty documentation, most of which can also be translated for end-user doc. Here’s my attempt at translation for end-user docs.

Scott says “Write stable, not speculative, concepts.” Scott’s talking about writing conceptual information only when its factual and can be trusted. For end-user doc, sometimes writing the concepts out helps you uncover problems, so I would argue that it’s better to write some concepts even early on.

Scott says “Prefer executable work products over static ones.” Scott’s talking about using test executables for code rather than documenting the code and its limitations. For end-user doc, I think that an executable work product could be a screencast or live demo sandbox that lets people try out a feature safely, or perhaps having a link in the online help that opens the dialog box in context like Windows Help in Windows 98 that would open the Control Panel being described in a task about setting display resolution.

Scott says “Document only what people need.” Amen, brother, preaching to the minimalism choir. Know your audience before you write down words that will be cruft.

Scott says “Take an iterative approach.” All writers know that re-writing is as essential as writing.

Scott says Be concise.” Agreed.

What are your thoughts about calculating documentation cruft? Are we taking it way too seriously, or is there a good tool in this (possibly crufty) post?

Subscribe to the comments for this post