Glom Python Documentation

I’ve done some work to document Glom’s Python API. Here is a temporary snapshot, though I will try to get it onto for the future. I used Sphinx, which is now used for Python itself. I’m fairly happy with it, though it’s not everything I hoped for.

What is the Glom Python API?

Glom lets you use Python to write field calculations (so field values can be based on other field values) and buttons scripts (stuff that happens when you press a button). That Python can code use the record and ui objects that Glom provides.

That Glom API is implemented by the pyglom (actually pyglom_1_14 currently) Python module. We use boost::python, writing the docstring comments in our C++ code, like so.

Bad Tools. Bad Syntax

Because it’s just a Python module, with docstrings, we can use standard Python documentation tools. However, I am not impressed by those tools. I am not impressed by Python documentation in general, and I suspect that the tools and conventions are the problem.

I understand that many developers don’t see the need for Python documentation because the arguments and return types for Python functions could be of any type and behaviour at runtime, but I believe in a deterministic universe, I know that most Python functions actually behave in a predictable way, I think that should be documented, and nobody can seriously write an application if they are really meant to doubt the behavior of the APIs they use. So let’s assume that Python APIs should be documented properly.

In the end, I had pretty good results with Sphinx, particularly after writing lots of documentation in its reStructuredText format. The output from pydoc was quite awful, though that’s partly because Glom’s python module is created with boost::python, which has some docstring issues and limitations.

But the main problem is that I can’t find any convention for documenting the arguments, return types and possible exceptions of Python functions. Various things (PEP 257, PEP 287,  reStructuredText, Sphinx’s reStructuredText primer) have been written about docstrings, but none of them mention this fundamental issue. So, these things are typically not documented, or are documented in a freeform unreliable way.

For comparison, here’s an example with doxygen for C++, using the simple Javadoc-style syntax:

/** A brief description in the first sentence.
* A more detailed description. This can be multiple sentences.
* Parameters such as @a boo can be mentioned here too, and
* mentions of an other_method() or an OtherClass will show up
* as links in the HTML. The parameter types and return type
* will be linked too.
* @param boo The goo's foo.
* @param hoo The voodoo that you do.
* @result The current moo.
* @throws FooError
* @deprecated Use some_other_method() instead, which does the necessary foo and bar.
Moo some_method(const Boo& foo, Hoo hoo);

gtk-doc uses very similar conventions, aiming to document the same aspects of APIs, for C.

doxygen will even automatically create links to classes and functions that you mention. sphinx’s reStructuredText requires awkward syntax such :class:`Record`, which also unnecessarily shows the module name in the resulting HTML.

Note also that Sphinx doesn’t default to using docstrings anyway. It seems to want us to write API documentation in separate files. I’m convinced that it’s a mistake – I’ve only known APIs to be properly documented when the documentation is directly in the code as comments, for instance via doxygen or gtk-doc. Luckily I could use docstrings via Sphinx’s autodoc module, though that doesn’t support all of reStructuredText, such as section headings.

I also like how doyxgen or gtk-doc documentation generally lists the functions at the top, with brief descriptions, allowing me to click on a function’s link to jump to the detailed description. Instead, with the spinx-generated HTML, we have to scroll through all the details of the whole class. And all the module’s classes are on one HTML page, obscuring things even more. Maybe there’s some way to fix these issues via Sphinx configuration.

8 thoughts on “Glom Python Documentation

  1. I myself had found python documentation much more readable and less redundant than the others.
    But in your case, giving better names for the function variables might help a lot.

    like here:
    show_table_details( (UI)arg1, (str)table_name, (object)primary_key_value) -> None

    arg1 is a really poor choice for a variable
    table_name bearing the suffix name is quite clear to be a string and i don’t think anybody would think something else
    the last param is suffixed with “value” but should be an object? Any specific object or just some class derived from object?

    So i’d suggest starting there.

  2. tm, yes, that’s not the real argument name. That docstring is generated by boost::python, as mentioned in the email that I linked to. See “boost::python, which has some docstring issues and limitations.”

    However, I don’t agree that good argument names are nearly enough. That still leaves people guessing.

      1. However, now that I’m trying this, I notice that:
        – It’s not really meant for use in docstrings. You’d have to use this trick to avoid two function signatures appearing in the html, forcing you to write signatures by hand for _every_ method:
        I wish it would just use the type: lines to improve the existing function signature.
        – When using boost::python, you have to use docstring_options objects to disable the generated function signatures that it generates. But this is only possible per-class, not per-function. These are not perfect but they do show some useful parameter type information. Ideally, the boost::python::arg() lines would optionally take a parameter type and description, then outputting everything in reStructuredText format. Maybe this is reasonable now that sphinx is the standard for Python.

          1. Thanks. I notice that you don’t use the “.. function” line, though I seemed to need that to make sphinx recongize the param lines. I’ll try again tomorrow.

Comments are closed.