?

Log in

No account? Create an account
entries friends calendar profile Elf Sternberg's Pendorwright Projects Previous Previous Next Next
Post-mortem: What I learned writing git-linter - Elf M. Sternberg
elfs
elfs
Post-mortem: What I learned writing git-linter

My latest contribution to the world is Git Lint, a plug-in for git that allows you to pre-configure your linters and syntax checkers, and then run them all at once on only the things you’ve changed. About half of us still live in the command line, and I like being able to set-and-forget tools that make me a better developer.


Here are a few things I’ve learned along the way about Python projects.


1. Use A Project Template


Project templates provide a means to magically produce a lot of the boilerplate you’re going to be producing anyway. I’m fond of Cookiecutter. While Git Lint started life as a single Bash script (later, a Hy script, and now a Python module), at some point I needed much more than just that: I needed documentation and testing. Up-to-date templates provide you with up-to-date tools: Tox, Travis, Sphinx, PyTest, Flake8, Twine, and a Makefile come pre-packaged with Cookiecutter’s base template, and that’s more than enough to launch most projects.


2. Setup.py is a beast


Getting setup.py to conform to my needs was a serious pain in the neck. It still doesn’t work correctly when installing to Mac OSX because the Python libraries and the manual (man pages) are in two different locations. If I’m building a command line tool, I always try to provide man pages. It’s usually the first place I look.


The manual pages also didn’t show up reliably in the build process; I had to force it by adding it explicitly to the manifest, even though it included the docs tree by default.


Setup.py and man pages are NOT friends.


Getting the build to include man pages, which I require for any command-line utility, was truly a pain in the neck, and now every upload to pypi has a manual step where I figure out if I have a man page to deliver or not. It’s truly painful.


3. Sphinx is a pain in the neck — but it’s worth it for Github Pages


Sphinx, the documentation tool for Python, uses RST (reStructured Text), which has just about the worst imaginable syntax for external links I’ve ever wrestled with. Inconsistencies about mixing links and styles drove me out of my mind.


On the other hand, I now have what I consider to be a solid idiom for generating Github Pages (gh-pages) from Sphinx documentation. A branch named “gh-pages” that contains your documentation will automatically be converted into a documentation tree on Github Pages (github.io), and you can see the results for Git Lint. This tree looks completely different from your development trees, so don’t get them confused, merge, or rebase them!


Simple generation of gh-pages


If you check out the Makefile, you’ll see the idiom clearly: it checks out a complete copy of itself into a sub directory, builds the documentation, copies it back to the parent directory, and then fixes all the links (because Github Pages really doesn’t like underscores in file and directory names). There’s irony in that it uses Perl to do the fixing– it’s just what I knew, it was fast, and I always have both Python and Perl installed.


This, by the way, points to another issue: always use the same virtual environment wherever you work. My Macbook and my Linux box had different versions of Sphinx on them, and the resulting generated pages were different on both boxes, making git report “everything’s changed!” when I went to fix a single typo in a link somewhere (I told you I hated those links).


It might be worth it to bag sphinx as a docker feature, or ensure that the version is locked down in your virtual environment.


4. Tox is amazing


Working with tox allowed me to be reassured that my code ran correctly every time, the first time. It did not catch other critical issues with installation, like the man page issue mentioned above, and that was painful to manage, but it did everything else.


5. Git Porcelain Zero is ridiculous


If you’re not familiar with git --porcelain, it’s an argument that many of the status-oriented git commands have that changes the output to a stable, machine-readable form meant to be consumed by other tools. Git Lint uses it a lot.


But the git --porcelain command doesn’t have any other guarantees: it doesn’t guarantee filename sanity, or unicode compatibility. For that, there’s git --porcelain -z, which produces a report in which everything is null-terminated so weird filenames can be consumed. This would be fine if the output were columnar, but it’s not always. The most egregious example I found was git status --porcelain -z, which is usually three columns, but if there’s an ‘R’ in the first column, then it’s four columns– ‘R’ means the operation is ‘rename,’ and the fourth column is the original name.


Since the -z argument makes both the cell and the line terminators null, you have to parse positionally. And if you’re parsing positionally and the number of positions can change, well, that’s context-sensitive parsing. And it’s ridiculous to have to put a context-sensitive parser into a small project like this. There was only one exceptional case here, so it’s a small issue, but inconsistencies like this really bother me.


6. Git lint is amazing


Now that I’ve actually used my little beastie, I can’t tell you how happy I am with it. As a full-stack developer with Python, C++, XML, HTML, CSS, Javascript, and some in-house stuff I can’t discuss, being able to check the entire toolchain without caring about what I’m checking, just set and forget, makes me extremely happy.


All in all, this was one of those projects where I learned a lot about everything: git, python, unit testing, documentation, github, jekyll, reStructuredText, Cookiecutter, PyPi. All this knowledge poured into one small project.

1 comment or Leave a comment
Comments
tracyabagwell From: tracyabagwell Date: October 5th, 2016 10:19 pm (UTC) (Link)

Question

▶ This, by the way, points to another issue: always use the same virtual environment wherever you work. My Macbook and my Linux box had different versions of Sphinx on them, and the resulting generated pages were different on both boxes, making git report “everything’s changed!” when I went to fix a single typo in a link somewhere (I told you I hated those links).◀


Different versions.... Maybe that's the reason why I am having problems. On my cell phone, I use Google..... I see a whole different look on, LJ and Facebook.

When I. Use factory set chosen pin app. It completely different look....

1 comment or Leave a comment