API techpubs tools work

API Archaeology: Complexity and sizing of an interface

For both OpenStack and Rackspace cloud APIs, we use WADL, Web Application Description Language, to build an API reference listing for all REST API calls. In a previous post I discuss how the reference pages at http://developer.openstack.org/api-ref.html are made with WADL source and a Maven plugin, clouddocs-maven-plugin. You can see the output for the Rackspace API reference page at http://api.rackspace.com, built with the same tool chain but different branding through CSS. I can discuss the tooling decision process in another post, but let’s talk about ongoing upkeep and maintenance of this type of API reference information.

In this post I want to dig beneath the surface to discover how complex these APIs are, how that complexity might translate into difficulty or time spent in documenting the interfaces, and discuss some of the ways you could assign the work for creating and maintaining reference information for APIs. In another post I said, start with a list. This post looks into what happens after you have a list and need more lists to know the shape and size of your API and its documentation needs.

Some of the complexity also lies in documenting the parameters and headers for each API. Just like unearthing the walls of an ancient structure, you can look at the various ways an API is put together by looking at the number of calls, the number of parameters on each call, whether there are headers on any given call, and how the calls are grouped and related. I’ve summarized some of that below for the comparison cultures, er, grouped APIs.

pedroszflickr

Definitions

A call is defined as a GET PUT POST DELETE command sent to a resource. These are known as HTTP verbs.

A header is defined as an optional or required component of a HTTP request or response. There are plenty of standard headers, what I’m talking about here are the extra headers defined by the API you document specifically.

A parameter may be a query parameter or provide a way to filter the response for example. Parameters specify a varying part of the resource and your users need to know what parameters are available and what they can do with them.

Running the numbers

To get these numbers, I first built each reference site so that the WADL files can be built into a single folder, which lets me do a grep for a count.

So I cloned each repo, ran mvn clean generate-sources with in the api-ref directory, then ran this command from with in from within api-site/api-ref/target/docbkx/html

grep -c “rax:id” wadls/*.wadl | sort -n -k2 -t:

Then I imported the output from the command as a colon-delimited file to a spreadsheet to get these counts.
OpenStack API Reference Metrics
Number of Compute v2.0 calls:290
Number of Networking v2.0 calls:92
Number of Orchestration v1.0 calls:41
Total documented calls:755

Rackspace API Reference
Number of Cloud Files calls:21
Number of Compute v2.0 calls:70
Number of Networking v2.0 calls:18
Total documented calls:670

Here’s a breakdown for just a few of the OpenStack APIs header and parameter counts.
Object Storage API Parameters: 12, Headers: 75
Volume API Parameters: 23 Headers: 0
Compute core API Parameters: 69 Headers: 1

Other metrics

We track doc bugs for the OpenStack API reference in Launchpad with the openstack-api-site project. There are nearly 200 doc bugs logged against the API Reference right now.

The three APIs with the most calls for Rackspace are Monitoring, Email & Apps, and Load Balancers, all of which are not OpenStack APIs. So a full two-thirds of Rackspace calls are not OpenStack-sourced. However, this means that a full third of Rackspace calls are identical to OpenStack.

What are some of the differences between OpenStack and Rackspace?
Extensions are complete in OpenStack; Rackspace only implements a handful of extensions.

Internal (admin-only) and external (user) calls are documented in OpenStack; Rackspace API Ref only documents external calls.

Rackspace has paid API writers and accepts pull requests on Github; OpenStack docs are written by writers and developers in the community (often with corporate sponsors) using the OpenStack gerrit process.

Conclusion

So that’s a lot of numbers, but what’s your point? My point is that making lists helps you determine the size and complexity of documenting multiple APIs. Not all companies or projects will have more than one API to document, but as we move towards more application interfaces for more business reasons, I believe that writers and developers need to get really accurate in their estimations of just how much time to allocate to document their APIs and do it well. Since these estimates are for API reference information only, don’t fail to also estimate time to write and maintain viable, tested example code as well. That’s a post for another day, thanks for reading about the complexity and comparison of OpenStack and Rackspace cloud APIs.

content strategy techpubs

How to get started writing API docs

I know a lot of people who want to consume awesome API docs. Let’s talk about what it takes to get started writing them. I’m not talking about completing your API docs. I’m talking about just getting started, what does it take?

For API documentation especially REST APIs, I’d advocate a reference-first approach. Like the couch to 5K program for running, let’s start API documentation on your couch. You look under your coffee table and find your shoes. You pull them on (make sure you have wrinkle-free socks!) and lace them up. You’re ready!

 

https://www.flickr.com/photos/jypsygen/3321559694/sizes/l/in/photolist-64vRYA-82CrNJ-86fGxc-9qLHZf-e8mSA1-acUfDf-kpF6vK-jKgd2h-jC5u26-ktM7vp-kpEjtn-kvG8qZ-cymDph-fMuVKS-7WuzPS-9V1uGW-8NfZuz/
Please write API docs! Photo courtesy jypsygen on Flickr.

If you are working on REST APIs, I’d also recommend understanding how they’re designed. I like the book, “A Practical Approach to API Design:From Principles to Practice,” by D Keith Casey Jr. and James Higginbotham. You can read a sample for free online, and then support their efforts by buying a copy at a price you set!

If you work is with another style of API, be sure you understand the underlying reasons for using that interface. It’ll help you understand your audience first.
Make a list. First, a list of all the calls. Then for each call, make a list of requirements. What must a user give to the interface to get back what they want? What are the requests, what are the responses? Then list the optional parameters the call can take. Are there any headers that can be sent or received? Be sure to write those on your list also.

Let’s not get tied up in tools yet, this initial writing work can be on a notebook or any text editor. You don’t run out and buy an expensive heart monitor or activity sensor when you’re just starting out as a runner. Figure out how far you can get with a pair of shoes (your notebook) before investing in cool tools and gear. Otherwise, you’ll get distracted by the cool tools and gear and not write stuff down!

With these lists, you’re building scaffolding. Just like a running program, you need to make a pattern to learn to pace yourself and spend your energy wisely. Once you have a list of the calls, you can write or diagram the users, the tasks they want to complete, and then see if your reference is filled in for all the users and tasks. I highly advocate the reference-first approach as it’ll help you test the completeness and helpfulness of your documentation.

I have some additional posts I’m writing for this year where I want to dive into increasing complexity of APIs so that writers and develeopers can estimate the amout of time needed for good documentation. I’ll also analyze some tooling for REST API documentation and offer benefits and tradeoffs for different tools. Looking forward to digging into API documentation in 2015!

community sxsw work

Hackathons of Late

I’ve participated in a few hackathons in the past year, since I’ve been working on a team of developer advocates at Rackspace. I wanted to chronicle some of my experiences with a few goals in mind: recording memories for myself, letting others get ideas for the wide range of hackathons, and chronicling the wide variety of types of hackathons and outcomes of hackathons.

For a bit of context, I’m female, have a family and weekend obligations that include sports practices, and I’m a generation-Xer if that gives you a sense of my age (with a range). I’ve definitely read and enjoyed a lot of articles about women and inclusive hackathons, “Running an Inclusive Hackathon: How to get better representation at your hackathon and recently about “grownups” and hackathons, “Why I don’t like hackathons, by Alex Bayley aged 39 1/2. I appreciate a well-run, inclusive hackathon for my own participation, but I also see a need for well-run, inclusive college-age hackathons.

Internal Hackathon at Rackspace

Screen Shot 2014-12-22 at 4.50.30 PM
This was a bunch of fun for me to dig into one of our SDKs (Python and pyrax) and services (Cloud Files) to make a gallery of photos from a balloon photography project we did for a girls in engineering day at the University of Texas. This project has had some re-use in demonstrating to school-age students about what code for the cloud looks like and it offers a nice demonstration since it’s visual. I sat next to another developer advocate who helped me break the idea into parts that I could manage more easily with my rudimentary coding skills. It was a ton of fun and at the end of the day I didn’t demo it for a group but did show it to our vice president who was walking around to see what we were working on. The hackathon was held in the office after another technical event during work hours, so it was easy to make it part of my day.

Austin Ladies Hackathon

This one involved a lot of time from a fun event Friday night to form teams and included hacking on two weekend days. I served as a mentor for teams in this hackathon rather than participating myself. It wasn’t the time investment that prevented me from participating, it was just my role to mentor rather than hack. I have a write-up on the Rackspace Developer Blog.
Austin Ladies Hackathon

Slashathon at SXSW Interactive

This event was one of the highlights of my week at SXSW Interactive, as a bookend with a really neat experience interviewing a woman on the StartUp Bus, Nicole Dominguez.

After the last day of SXSW Interactive, we all arrived at Capitol Factory in the morning for the hackathon. I was toting a Rackspace red bike and scheduled to be a developer mentor for the Rackspace Cloud APIs for the morning session.

IMG_2444

Everett Toews and I both work on the developer relations group at Rackspace, and he gave a presentation about how to hack the hackathon by using our cloud services. The other presenters really piqued my interest in metadata about music, bands, albums, and artists headphone hardware advances (hacking headphones are amazing!), cool uses of location and proximity hardware devices, all of the technology really makes ideas fire off in your brain. We came up with an idea collaboratively with four of us – Everett, and two web developers from Harvard Business Review who had to catch afternoon flights. We wanted to create a site for seeing on a map where your favorite band or musician is, with related Tweets and social media posts and the ability to buy tickets for their next gig. Everett presented to the panel of judges – Slash himself, Robert Scoble, and the guy who invented bittorrent, Bram Cohen. It went over really well! The winner had some cool hardware that helped enhance a concert experience and second place went to a Google Glass app that performers could wear to measure the decibels of audience noise. LoudWire covered the event in a post.

Types of Hackathons

I definitely see a difference between hackathons for college-age or students and for working adults or non-traditional students. There are also hardware hackathons versus software hackathons or hackathons with a specific theme, such as the Slashathon that featured music or audio hardware and music-based APIs. There should be a hackathon for many interests, then it’s a matter of determining your own goals for participating.

Outcomes of Hackathons

I think there are different types of outcomes: projects by participants, learning by participants, and then for sponsors and mentors, recruiting and learning more about participants or testing their products. The outcomes I don’t like to see are companies trying to get hard work for “free” or with little investment, or participants focusing too much on winning prize money.

What do you think about hackathons lately? What are your experiences? I’d love to hear more from others experiencing hackathons as a participant and as a sponsor and mentor.

blogging writing

More posts around the web

Seems like more and more I’m writing everywhere but here! I wanted to do a round-up of some of my writing around the web.

On the Rackspace Developer Blog: Austin Ladies Hackathon

This weekend was a blast, hacking with all women at Rackspace, and wow were the projects impressive. Here’s how it went.

Friday night we gathered at a restaurant, the Flying Saucer, to meet each other and start to form teams. I’d guessimate that 20 of the 30 participants showed up. I helped out by facilitating discussions for people to find teams, and I spent a lot of the night talking to complete newcomers about what a hackathon is like, what to expect, and especially how to learn at one. We also recruited for the upcoming NoSQL Mobile App Challenge (http://rackspacemobile.challengepost.com/). (Read more …)

On the O’Reilly Radar blog: The book sprint: Not just for code any more

Do you really want a technical book for your project? Does your community need to provide more helpful docs to support even more users? Does your community have a lot of knowledge they need to get out of heads and into bits and bytes? Do you have a good mix of technical experts and technical writers and users who would enjoy each other’s company for a week of hard work?
If the answer is yes, then consider a book sprint. (Read more…)

On the opensource.com blog: The Women of OpenStack talk outreach, education, and mentoring

In the open source world, a women-only event seems counter-intuitive. Yet I am finding reasons for such events the more I attend them.

At the OpenStack Summit, a twice-a-year event where OpenStack contributors get together to plan the next release, the Women of OpenStack group has set up events where we invite the women first. Men aren’t excluded, but our hope is to get more OpenStack women together. I can hardly capture the value of getting together with other women in OpenStack at the Summit, but here goes.(Read more…)

work writing

All About that OpenStack Operations Guide

I wrote a post on the O’Reilly Programming blog about our experiences writing a book in five days and revising it for publication as an O’Reilly book. The blog post is titled, The book sprint: Not just for code any more. This infographic summarizes the numbers quite well.
Many thanks to Racker Claudia Charbel for this infographic and for showing off the capabilities of Canva.com!

booksprintinfographic

community techpubs tools

Tearing down obstacles to OpenStack documentation contributions

Rip. Shred. Tear. Let’s gather up the obstacles to documentation contribution and tear them down one by one. I’ve designed a survey with the help of the OpenStack docs team to determine blockers for docs contributions. If you’ve contributed to OpenStack, please fill it out here:

https://docs.google.com/forms/d/136-BssH-OxjVo8vNoOD-gW4x8fDFpvixbgCfeV1w_do/viewform

barriers_sameleighton
I want to use this survey to avoid shouting opinions and instead make sure we gather data first. This survey helps us find the biggest barriers so that we can build the best collaboration systems for documentation on OpenStack. Here are the obstacles culled from discussions in the community:

  • The git/gerrit workflow isn’t in my normal work environment
  • The DocBook and WADL (XML source) tools are not in my normal work environment
  • My team or manager doesn’t value documentation so we don’t make time for it
  • Every time I want to contribute to docs, I can’t figure out where to put the information I know
  • When I’ve tried to patch documentation, the review process was difficult or took too long
  • When I’ve contributed to docs, developers changed things without concern for docs, so my efforts were wasted
  • Testing doc patches requires an OpenStack environment I don’t have set up or access to in a lab
  • I think someone else should write the documentation, not me
  • I would only contribute documentation if I were paid to do so

Based on the input from the survey, I want to gather requirements for doc collaboration.

We have different docs for different audiences:

  • cross-project docs for deploy/install/config: openstack-manuals
  • API docs references, standards: api-site and others

These are written with the git/gerrit method. I want to talk about standing up a new docs site that serves our requirements:

Experience:
Solution must be completely open source
Content must be available online
Content must be indexable by search engines
Content must be searchable
Content should be easily cross-linked by topic and type (priority:low)
Enable comments, ratings, and analytics (or ask.openstack.org integration) (priority:low)

Distribution:
Readers must get versions of technical content specific to version of product
Modular authoring of content
Graphic and text content should be stored as files, not in a database
Consumers must get technical content in PDF, html, video, audio
Workflow for review and approval prior to publishing content

Authoring:
Content must be re-usable across authors and personas (Single source)
Must support many content authors with multiple authoring tools
Existing content must migrate smoothly
All content versions need to be comparable (diff) across versions
Content must be organizationally segregated based on user personas
Draft content must be reviewable in HTML
Link maintenance – Links must update with little manual maintenance to avoid broken links and link validation

Please take the survey and make your voice heard! Also please join us at a cross-project session at the OpenStack Summit to discuss doc contributions. We’ll go over the results there. The survey is open until the first week of May.

community work writing

How to Build OpenStack Docs and Contributors through Community

I’m well past the three year mark, working on a new open source project that grows and grows every six months. I’ve been working closely with Diane Fleming at Rackspace to focus completely on upstream OpenStack. Upstream means that all of our documentation work goes to the open source project itself. So while Rackspace runs OpenStack in production and for our customers private clouds, Diane and I focus on documentation that helps any organization run and use OpenStack. We have put together an outline of what we do to make upstream OpenStack documentation better all the time.

Growth, scaling, and related challenges

When I started, there were just two projects with two APIs. Now we have 130 git repositories fostered by over twenty related programs. As you can imagine this scenario causes scaling difficulties but we are bravely making our way. Here are some of the challenges we have faced and what we’ve done to lessen the pain of coordinated, collaborative technical documentation in an open source community.

conversationplus_black_white

In the face of language and technical barriers, the OpenStack docs team used a combination of IRC meetings, documentation boot camp, Google hang-outs, a busy docs-team mailing list, and other methods to create a flourishing, global team of writers and technical contributors who have immensely improved the OpenStack docs in the last couple of years. The release that went out in Spring of 2013, three people wrote half of the docs. For the release that went out in Fall of 2013, seven people wrote half the docs. I can’t wait to see what our numbers are for the release going out next, on April 17th.

Team-building tools – the good, the bad, and the ugly

Here are some benefits and pitfalls of these tools:

  • IRC: Pros – clear agenda, follow-through week-to-week, global participation now that we have APAC and North American meetings. IRC meeting bogs enable an automated log of minutes. Cons: difficult to find agreeable time, no face-to-face, hard to introduce new topics.
  • Office hours: Seemed like a good idea but fell by the wayside, we did not have much attendance. As our team grows, people can stop by the IRC channel at any time to get one-on-one help.
  • Google Hangouts: With the video and voice enabled, it is nice to see each other’s faces without having to travel. Hard to find an agreeable time with the round-the-world team.
  • Boot camp: Extremely positive experience – spawned new ideas and new connections. Downside is the cost/time factor. We had great survey responses but decided we didn’t need one every six months.
  • Mailing list: Good way to resolve immediate issues and gather consensus as well as multiple view points.
  • Book sprints: Good way to get a needed book written and distributed. Not a good way to build ongoing community for maintenance, and in some ways you have to be careful not to build a book that no one else thinks they should contribute to. But community is just one part of good docs – this is a good complement to other efforts.

OpenStack docs – before and after

How have the docs changed due to team building?

  • Cleaner, more compact library. We did a huge refactor prior to the Boot Camp, which has made it easier to specify what types of content goes where.
  • Better writing. Professional technical writers have done an amazing job avoiding “frankendoc” — by editing, reviewing, and polishing with an agreed-upon style guide, we improve the actual writing to better serve readers and users.
  • Better technical content. In OpenStack, teams have core reviewers and most teams require two core reviewers must approve a change before it gets built to the published site. As we expand our reviewers (not just core but many reviewers) our docs have improved technically.
  • Better automation. By writing tools that scrape the code for docstrings we are able to keep up with fast-moving projects that release every six months.
  • Timing with releases. We carefully scope what documents are considered tied to a release. The Install Guides and Configuration Reference are the only two books built from a set release. All other documents are continuously published.

OpenStack contributors – before and after

How have the relationships among contributors and contributors’ roles changed due to team building?

  • Much greater communication among contributors – now we know each other on a more personal level, and feel more comfortable working together
  • Contributors have found where their strengths lie in the community. Some people are more tools and gear heads and build gates and tests and build tools. Some people are natural editors and review heavily with suggested edits. Some people are blue-sky visionaries. Some people are heads-down system administrators and architects. There’s a place for everyone, not just writers.

What’s to come?

We want to discuss revisions to our original vision openly. This blog post is a starting point, but we are listening on all available channels. At the OpenStack Summit in Atlanta in May, I want to collaborate on a request for proposals for a new front-end design for our documentation that can help us make changes to how docs are authored. I’d like to find out more about ways to enable non-CLA contributors to the docs. I have lots of ideas and look forward to working with this amazing group to improve our processes and results.

conversation_black_white

community work

Finding an OpenStack Mentor

Last week I ran an internal “So You Want to be an OpenStack Contributor?” workshop showing the different ways to work on OpenStack. Here’s the slide show so you can see the way I approached it. As the Documentation Program Technical Lead you’d think I’d steer people straight to the documentation bug backlog, but I try to find out where interests lie before going straight to doc fixes. Definitely people should read the docs as a great start.

You can work on OpenStack in non-code ways, such as bug triaging. Also the OpenStack Foundation does community marketing and staffs booths at events from the community. But a great way to understand the ins and outs of OpenStack-landia is to commit a patch.

I have to admit, I didn’t know much when I first started working on OpenStack at Rackspace. The Swift team was the group I had immediate access to in person. Wow were they patient with me while I made hilarious-in-hindsite errors. I had a patch where I changed “referer” one r to “referrer” two rs, because duh that’s how referrer is spelled. Well as it turns out that’s not the way the WC3 HTTP Protocol specifies request headers since 1992 or so, woops! Then I also managed to change the RST backticks (`) to single quotes (‘) which is absolutely not going to render correctly with Sphinx. Chuck Thier patiently explained the errors I had made and how to correct them. So do not be discouraged if it’s difficult to get the hang of your first patch or two or ten. Code reviewers are happy to help you iterate and revise. I’ve heard of good and bad patch reviewing going on in the community so I encourage you to find a real person who can help you get helpful reviews.

We also have organized OpenStack mentor programs now. We’ve been participating in the GNOME Outreach Program for Women for three rounds, and we’re a participating organization with the Google Summer of Code program for 2014. There are ideas for projects on the OpenStack wiki:

We have dedicated IRC channels for new contributors – #openstack-101 and #openstack-opw on freenode. Our OPW interns have written great blog entries about getting started with OpenStack (In a nutshell:how OpenStack works) and DevStack (Installing DevStack with Vagrant). Their fresh eyes make for great starting points. I encourage us all make this work both ways – people of OpenStack, be mentors, and newcomers, seek out the people in OpenStack who want to help you get started. Updated to add: be sure to check out opensource.com for “How to Contribute to OpenStack.”

community techpubs work writing

OpenStack Operations Guide Mini Sprint

oreilly-openstack-ops-guide

We held a two-day mini-sprint in Boston at the end of January to update the OpenStack Operations Guide. You may remember the first five-day sprint was in Austin in February 2013. This time, the sprint was shorter with fewer people in Boston and a few remote, but we had quite specific goals:

  • Update from Folsom to Havana (about a year’s worth of OpenStack features)
  • Roadmap discussion about nova-network and neutron, the two software-defined networking solutions implemented for OpenStack
  • Add upgrade instructions from grizzly to havana
  • Implement and test the use of parts to encapsulate chapters
  • Address editor comments from our developmental editor at O’Reilly
  • Add a reference architecture using RedHat Enterprise Linux and neutron for networking

Some quick wins for adding content were:

We added and updated content like mad during the two days:

The two toughest updates are still in progress, and our deadline for handover to O’Reilly is this Wednesday. The first tough nut to crack was getting agreement on adding an example architecture for Red Hat Enterprise Linux. We are nearly there, just a few more fixes to go, at https://review.openstack.org/#/c/69816/. The second is testing the upgrade process from grizzly to havana on both Ubuntu and RedHat Enterprise Linux. That’s still in progress at https://review.openstack.org/#/c/68936/.

The next steps for the O’Reilly edition are proofreading, copyediting, and indexing over the next six weeks or so. I’ll be keeping the O’Reilly edition in synch with our community-edited guide. As always, anyone in the OpenStack community can contribute to the Operations Guide using the steps on our wiki page. This guide follows the O’Reilly style guide rather than our established OpenStack documentation conventions. I’m looking forward to a great future for this guide and we’re all pretty happy with the results of the second mini-sprint.

Thanks to everyone making this a priority! Our host at MIT was Jon Proulx joined by Everett Toews who braved airport layovers and snow, and Tom Fifield who wrote the most patches despite a complete lack of sleep. Joe Topjian worked on edits for months leading up and has been tireless in making sure our integrity and truth lives on through this guide. Thanks too to the the hard working developmental editor at O’Reilly who offered lunch in Boston, Brian Anderson, joined by Andy Oram. David Cramer got DocBook parts working for us in time for the sprint. Summer Long worked long and hard on the example architecture for RedHat. Our remote reviewers Matt Kassawara, Andreas Jaeger, and Steve Gordon were so valuable during the process and ongoing. Shilla Saebi gave some nice copyediting this past week. What an effort!

community content strategy techpubs tools

A Few of my Favorite Things for 2013

tumblr_m8lsqnjijt1qgx2ojo4_250

This year has been filled with interesting finds, discoveries, and productivity. Plus oxford commas! Here are my favorite things for 2013.

The Hunger Games trilogy, because it’s like a window into a mind of a smart writer who writes with purpose.

The Documentation chapter of the Developer Support Handbook has to be one of my very favorite things I discovered this year. I’m on the Developer Relations Group team at Rackspace and this is a great handbook for all of our team.

Animated GIFs, pronounced jifs, am I right? OpenStack Reactions cracks me up.


Grace Hopper Conference by the Anita Borg Institute, especially the Open Source day, and the GNOME Outreach Program for Women which OpenStack started participating in this year. Women in technology are my favorite!

The Houzz App on my Android tablet for eye candy while messy remodeling was actually happening. Plus it’s the best content remix site I’ve seen in a while, more targeted than Pinterest.

Photo kids

Probably the best photo of my kids this year, I make it a favorite because at their ages it’s difficult to get one of the both of them.

OpenStack Docs Boot Camp

OpenStack Security Guide book sprint, read it at http://docs.openstack.org/sec/.

oreilly-openstack-ops-guide

OpenStack Operations Guide book sprint, now an O’Reilly edition, read it at http://docs.openstack.org/ops/.

How about you? What are some of your favorite things from this past year?