I wanted to document the migration journey while we are in the middle of it. Last week the OpenStack Super User site did a great article about the migration, How you can help with the documentation site refresh. We talked about the reasons for it: enable more contributors, offer simpler markup, be more Pythonic.
And more Pythonic we are. Let’s dig into what’s going on with the migration, what we’ve learned, and what we still have to learn.
Migration process
We’re nearly done with patches to review converted End User Guide files, but just a few Admin User Guide files are complete. Sign up on the OpenStack wiki at Documentation/Migrate. I’ve figured out how to include certain files in a build conditionally with an extension we’ll call “scope” for now. Using (and abusing) the meta information by putting a special directive in a file meant for admin-only lets us “tag” certain files for inclusion and build output with those files included.
How to build locally
We have always had tox jobs for building and testing the docs. The newest one runs the sphinx-build command for the end user guide.
If you don’t want to use tox, install these prereqs locally to test only the build:
pip install sphinx; pip install openstackdocstheme
Next, switch to the directory containing a conf.py and run:
sphinx-build /path/to/source/ path/to/build/
This command builds html output by default.
In troubleshooting some markup I found that the sphinx-build command does not give as much information about the markup as python setup.py build_sphinx does. So I will test a switch with a setup.py file as well as a conf.py file for each guide in the openstack-manuals repo.
To build locally with tox, follow the instructions on the wiki page. Building with tox is preferred as that matches our gate tests for patches. Feel free to ask for help in #openstack-doc on freenode IRC if you run into errors. We have tested the instructions on Linux, Mac, and Windows.
Conventions and editing RST markup
Docs tools guru Andreas Jaeger already had a head start in adding RST markup conventions to our Documentation/Markup_conventions wiki page on the OpenStack wiki. I’m finding that there are some types of content, such as extra information embedded in a list item like a table, that just can’t come over in its current state. I’m still working through these questions and it seems as though numbered lists can’t have much more than single list items. Anyone with info, please let us know if these are possible:
- How to get numbered list continuation to work when you have a table after a #. list item?
- How to get numbered list continuation to work when you have a bulleted list between #. list items?
- How to get get embedded .. note: directives to work between numbered list items?
Matt Kassawara asked if there is a side-by-side editor for RST like there is for Markdown, and so far there is not, but the Sphinx development mailing list has a thread about what authors currently do as well as what they’d like to build.
Priorities
We still need output bugs fixed. To me, the top priorities are:
- Sphinx template needs precise “Log a doc bug” link created on-demand similar to current functionality
- Sphinx openstackdocstheme needs to be tested for the translation toolchain
- Sphinx openstackdocstheme doesn’t style admonitions (note, warning, important) correctly
- Plus sign appearing when numbered list followed by bulleted list
What I need to understand is what would compel us to update Sphinx itself, considering we don’t know when their next release is. We already pin to a beta version (1.3b) so perhaps we can patch as needed and pin our version.
Troubleshooting
Generally, make sure your Python environment is installed and ready to go, and then use a virtualenv to be sure you have “corded off” the environment and know exactly what’s installed.
Use “pip freeze” to get a list of what is installed. If a package is missing, make sure you have access to pypi by running the pip install command with -vvv. Also if you’re not working in a virtualenv, ensure you run the install command with sudo on Mac or Linux.
If you’re working within a virtual machine like VirtualBox, you may find that networking settings change even if you go from wired to wireless or from one wireless network to another.
If the output looks strange to you, delete the /build/ directory then re-build. I suspect that CSS and JS files remain outdated or are not copied over every time.
Please log bugs in openstack-manuals with the tag “openstackdocstheme” if you see bugs in the output.
What this means for new guides in progress
The only new guide in progress is the Networking Guide and some of it’s in markdown, some of it’s in DocBook, and all of it can go to RST. We must get the translation toolchain working prior to publishing, and I’d prefer to get more output bugs fixed prior to switching any more guides.