Tuesday, June 26, 2007

Death to xfs

That's xfs the X font server, I have no beef with the xfs filesystem. We've been using xfs for core fonts[1] since Red Hat 6, mainly because we've had the chkfontpath tool for editing the xfs config file and restarting xfs. This tool lets us add and remove new font directories to the xfs config file and restarts xfs when necessary, so that a running X server will pick up new fonts as new font packages are installed.

While all that is nice, it's a pretty weak reason for running an system daemon, considering the maintenance overhead, security problems and boot time slowdown. So to get the best of both worlds, I wrote a patch to libXfont (which is what xfs and the X server uses for locating and rendering fonts) to let it pick up fonts paths in a rc.d style way. Instead of relying on a config file for the font path, you can now additionally redirect to a directory of symlinks, such as /etc/X11/fontpath.d. The symlinks point to the directories that make up the font path and the symlink names can include attributes such as :unscsaled. libXfont will automatically detect when links are added and removed and rescan the directory.

No, core fonts are not going away. We can't ever do that, to much legacy. But since Fedora have the two required core fonts ("misc" and "cursor") compiled into the X server, you can have a usable setup with no core fonts (say, maybe you don't like bitmap fonts, or maybe you're OLPC), no xfs, no chkfontpath installed. And at the same time, if you must have that belowed bitmap courier in your terminal or are stuck with an old motif app, install the core font you need and it will just work.

[1] Core fonts is the font mechanism originally designed into the X server. The X server locates and renders the fonts on behalf of the client. This requires the X server to know about parsing font files, glyph rendering and arbitrary encodings, which you typically would want to keep out of a priviledged process. Modern applications locate fonts using fontconfig, render glyphs using freetype, and only upload the resulting images to the RENDER extension.

Wednesday, June 13, 2007

Setting up a cloned git repo

I was going to set up a clone of the xserver git repo in my people.freedesktop.org home directory and I figured I'd document the steps. If you want the cheat sheet just skip to the last couple of paragraphs. First of all, to set up a repo I need ssh access to people.freedesktop.org, but since I was going to put the repo in my home directory there, I already have that. Now, the official git repos are read only mounted on /git, so the obvious thing to do is to say

krh@annarchy:~$ git clone --bare /git/xorg/xserver.git xserver.git
Initialized empty Git repository in /home/krh/xserver.git/
...
krh@annarchy:~$ du -sh xserver.git/
24M     xserver.git/

Most of this space is in the objects representing the files, directories and commits of the project history. Compared to the repo we cloned from, it's pretty efficient, as git compresses all those object into a pack as part of the cloning process:

krh@annarchy:~$ du -sh /git/xorg/xserver.git/
135M    /git/xorg/xserver.git/

But given that both repos are on the same filesystem, we can ask git clone to share the underlying objects. This is a pretty clever trick that uses the fact that git objects are immutable. Once you've stored a file or an entire revision of you project as part of a git commit, that object never changes. It would be nice if git could just detect this and do the right thing, but you have to pass a couple of options:

krh@annarchy:~$ git clone -s -l --bare /git/xorg/xserver.git xserver.git
Initialized empty Git repository in /home/krh/xserver.git/
krh@annarchy:~$ du -sh xserver.git/
1.3M    xserver.git/

That's a lot better though, we're down to 1.3M for my own little copy of the xserver repo. However, when you think about it, it's just a glorified symlink to /git/xserver.git and 1.3M is a pretty heavy symlink. It's pretty easy to spot the problem

krh@annarchy:~$ du -h xserver.git/
252K    xserver.git/refs/heads
932K    xserver.git/refs/tags
1.2M    xserver.git/refs
...

When we converted the xserver repo over, we of course imported all branches and tags, most of which now are just in the way—when was the last time anybody looked at xprint_packagertest_20041125? One solution is to just nuke the branches and tags you don't care about. But if you need to preserve this important historical information in your repo or are just to lazy to weed it out, you can say

krh@annarchy:~$ GIT_DIR=xserver.git git-pack-refs --all
krh@annarchy:~$ du -sh xserver.git/
124K    xserver.git/

Again, git should just do this by default in the same way it compresses the objects when cloning. As a final step, I want the gitweb script and the git daemon to pick up my new repo so I can browse it on gitweb.freedesktop.org (or cgit) and clone it using the git protocol. To do that I need to touch a special file in the repo:

krh@annarchy:~$ touch xserver.git/git-daemon-export-ok

which is just a way to say that it's ok to export this repository to the world. It typically takes a little while before the gitweb script finds your repo.

I'm sure there's somebody shaking their head now thinking, "gah, git is useless, look at all those commands", so to address that let me first sum up what you have to do to create your own repo and make it visible to the world:

krh@annarchy:~$ git clone -s -l --bare /git/xorg/xserver.git xserver.git
Initialized empty Git repository in /home/krh/xserver.git/
krh@annarchy:~$ touch xserver.git/git-daemon-export-ok

Second, this is my own repo, I can create all the branches I want and commit crazy stuff, without affecting the upstream repo. I set it up without bugging any of the freedesktop.org admins and I didn't even need xserver commit access. Third, if it turns out that I do useful work there, we can merge it back into the main repo, without loosing any history, and without polluting the upstream branch name space with branch names suchs as gah, doh and gahgah, which are among my favorite choices. Allthough, they'd fit right in with the rest of the xserver branches.

Thursday, June 7, 2007

Version Control

Everybody is blogging about SCM tools, and since I have this soapbox at my disposal I might as well too. I've been using git for over a year now, and I think I've earned the right not to have my preference dismissed as flavor of the week at this point. And I feel that the time I've spent learning git has been more than recuperated, and I'm sure anyone who sits down and learns git will feel the same way within a month or so. But OMG it has over 100 commands in /usr/bin, there's this thing called the index, sometimes you have to edit a text file, and hey I got this scary looking warning, bla bla... still, I'll claim—and I don't do this lightly—that git is the single most useful software development tool I've learned in the past 10 years or so.

CVS is shit. Subversion is gold plated shit. Some people claim that SCMs doesn't matter and that we should just keep using what "works". I hear this from people who typically don't do much development anymore, and certainly haven't tried anything other than CVS or SVN. If you truly don't care about SCMs, get out of the way and let people that care and have investigated the options pick a system that actively assists us in software development.

Centralized SCM advocates often think that it's either or; either it's the SVN/CVS model or it's the Linux model where everybody has his own tree, everybody pulls from everybody and nobody knows where the latest official version is. The way it works, though, is that most distributed systems provide a superset of functionality of centralized systems, and can be set up—through configuration and convention—to work with a centralized repository. This is how most of the freedesktop.org git repositories work.

People have been using cvsup, rsync or other mechanisms to copy and synchronize remote cvs repositories long before distributed systems were available. git can clone an svn repository to a local git repository where you can commit and branch, and then eventually push those changes back to the upstream svn repository. However, if you're choosing an SCM for your new project why would you choose a system that requires your developers to jump through these hoops to achieve that? Given that git can be set up with a central repository, ssh accessible to a group of developers just like cvs and svn, there is no reason to cripple your co-developers workflow, just because you feel that distributed SCMs have nothing to offer.

Another point often raised about distributed SCMs is that they encourage private development. No they don't. My CVS workflow when implementing a new feature used to be: hack on huge patch, break it, fix it, add more stuff, break it, fix it, etc until the patch was done. That is private development. Now, with git, I add a branch in a private copy, work on the feature in a series of commits, and if the feature works out alright, merge it to the upstream repository. During the entire process the branch is visible and the repository can be cloned by people who wants to test or contribute.

One tool I'd like to do if I ever get some spare time is a svn commandline compatible wrapper for git. It's entirely doable, and shouldn't be too much work. svn checkout becomes git clone, svn update becomes git pull, svn commit becomes git commit -a; git push. I'm sure the devil is in the details, but having this wrapper lets you choose a repository format that preserves branching and merging history and is designed to support cloning, while still letting people use the CVS/SVN workflow.