git and patches

Sometimes you want to send or receive patches of work done. I found this great tutorial on how you can do that using git, which I will briefly rewrite.

How to create patches using git?

I assume you have cloned the original repository, and created five commits that you want to send back to one of the developers. My favourite way to do that is by

git format-patch HEAD~5

The command will create five separate patch files, which you can attach to an e-mail and send. The ~5 means "the fifth commit before", in other words, the sixth last commit. That would be the last commit that are already in upstream, which you are not interested in. Assuming the work is done on the master branch and the remote repository is called origin, you could also do

git format-patch origin/master

which will do the same, create one patch file per commit since the commit which is already in origin/master.

How to receive patches?

The patches includes a header which contains the commit message, and author information. Normally you want to keep that. The initial thought is probably to use

git apply file.patch

However, this will only apply the changes, not commit them. Instead, one should first use the apply command to check that the patch is OK:

git apply --check file.patch

If you do not receive an error, it means that the patch will apply without failing. The way to apply it then is using

git am --signoff file.patch

This will commit the patch with original author information, and append to the commit message that it was signed off by you. The patches are numbered, starting with 0001. After you have committed all the patches to your own tree, you should make sure that they are in fact correct. If you find problems, you can always use rebase to remove commits again.

git rebase -i HEAD~5

will open an editor where you can select commits that you want to change. You can delete commits simply by deleting the corresponding line.

That’s it, now you can in an efficient and controlled way distribute and receive patches even when the repositories are not stored in a publicly available place!

Bought new mouse

I was in the store yesterday, and came across the Logitech M195 cordless for 15 CHF (about 18 USD). I figured a nice small cheap mouse would be nice to keep in my backpack for when the trackpad does not suffice (e.g. when I want to play games). The mouse comes with a tiny receiver, which can be stored inside the mouse when you do not use it. It is also so small that you can just keep it in your usb port without any worries that it might break or similar. Only issue I have with that is that my Macbook only have two USB ports, so that wont work for me. The mouse seems to track nicely even on fairly blank surfaces, and a piece of paper is more than enough for good precision. I tried it both in OSX and Linux, and it starts working immediately without any need to worry about drivers or other annoyances. Great value!

CERN and kerberos tokens

This is a short blog post probably only useful for people working at CERN.

For Linux boxes we use afs at CERN, with Kerberos as the authentication method. For people with the default SLC installation, this is set up nicely from the get go. For people like me that use another Linux distribution, or perhaps Mac OSX, it is something you have to set up for yourself.

Installing afs and kerberos is usually not a very compilcated task, and I will not go into that here. What I wanted to do, was to recommend on how to use it. The "kinit" command is used to get a kerberos ticket, and then one uses "aklog" to get the afs token (whatever this means, for me it is some security jibberish). Now you can actually add the flag "–afslog" to the kinit command and directly get the afs token as well. You should also add the "-f" flag to get a forwardable token for some reason.

There is a slight annoyance with the kerberos tickets, they are only valid for 25 hours at CERN. What I figured out today however, was that if you add the flag "–renewable" to the kinit command, then you can renew your ticket with "kinit –renew" without getting promted for a password.

Hence, what you should then do is something like this. Add first to your ~/.bashrc (or other rc if you use a different shell):

alias cernconnect="kdestroy && kinit --afslog -f --renewable user@CERN.CH"

Then, in your crontab ("crontab -e" in the terminal to edit), add the following line

@daily ID=afstoken kinit --renew

With this set up, you can just write "cernconnect" in the terminal and then write your password, and your computer will automatically take care of renewing your token every 24 hours. You can also add the flag "–password-file=/path/to/passwordfile" to kinit so that you do not have to write your password, but take care, if someone gets this file they get full access to your account! This should work on both Linux as well as OSX, though I have only tried it on Arch Linux for now.

Get bug tracker for your gitorious repository

First blog in quite a while this. I have recently been playing around with new tools, namely Apache, MySQL, Gitorious and lately Trac. Getting the http server running on my desktop was much easier than I first expected, and there are too many fun tools out there to try out once you have that going. Today I would like to write a tutorial on how you get Trac to work with your gitorious repository. This tutorial will include a setup that makes it possible to write commit messages that are automatically appended to a given bug/feature request, and even make it possible to close the ticket. I assume that you already have a git repository in a folder /path/to/your/git/repo. If it is not a bare repository, the path is /path/to/your/git/repo/.git/. I also assume you have a python 2 install on your machine.

Configuring Trac

First, you need to install Trac. On Arch Linux this is readily available in the package manager, and I suppose that is the case for most other distributions as well (correct, this tutorial is not for the Windows users of you). Further you need to install the GitPlugin from http://trac-hacks.org/wiki/GitPlugin. This can be done with the command

easy_install http://github.com/hvr/trac-git-plugin/tarball/master

This assumes you are using Trac 0.12. Have a look at the link for further information. Now it is time to set up a trac project. The trac project is a folder which will contain the configuration and information for that bug tracker. This folder can be moved around as you feel like (so backup is easy). Set up the environment with

trac-admin /path/to/folder/wher/you/want/trac/project initenv

This will ask you for the project title and what kind of database you want. SQLite is the default option, and a good one in my opinion since you then have all information contained in this folder. Now you open the folder /path/to/folder/wher/you/want/trac/project/conf/trac.ini where you need to add some options under new categories, namely

[components]
tracext.git.* = enabled
tracopt.ticket.commit_updater.committicketreferencemacro = enabled
tracopt.ticket.commit_updater.committicketupdater = enabled

[git]
cached_repository = false
persistent_cache = false

The git category is some specific options for the plugin. The components-category contains the plugins you want activated. This includes the CommitTicketUpdater plugin that is included in the default Trac installation. In the trac category you need to change the following options for the git plugin:

[trac]
repository_dir = /path/to/your/git/repo
repository_type = git
repository_sync_per_request =

Remember that the path to your repository should include the .git folder if it is a normal repository. Finally, for the CommitTicketUpdater plugin you need to add some parameters in the ticket category. This is further explained in the link above.

[ticket]
commit_ticket_update_envelope =
commit_ticket_update_commands.close = fixes
commit_ticket_update_commands.refs = <ALL>
commit_ticket_update_check_perms = false
commit_ticket_update_notify = false

This finalizes the configuration of Trac. Lastly you probably want a way for people to log in. What I chose (and it is probably a weak solution but it works), was to use htpasswd for this purpose. This is explained in the installation wiki for Trac. To add the first user I used

htpasswd -c /path/to/folder/wher/you/want/trac/.htpasswd <user>

and then for successive user accounts I dropped the "-c". You can then start your server with the command

tracd -p 8000 \
--basic-auth="*,/path/to/folder/wher/you/want/trac/.htpasswd,Some Credential" \
/path/to/folder/wher/you/want/trac/project

and open it up the link http://localhost:8000 in your browser. Note that you can have many Trac projects, just add the path to each project separated by space in this command. You probably want to create a small script that contains this command and call it something like "start" so that you do not have to manually write this command every time… Put it in /path/to/folder/wher/you/want/trac/.

Adding a hook to the git repository

Now, in order to get your commits to automatically append to tickets, you need to add what is called a hook to the git repository. This is essentially a program that is ran at a certain point during your interaction with git, for example every time you make a commit (pre-commit or post-commit). For a bare repository you find them in /path/to/git/repo/hooks, and for a normal repository you find them in /path/to/git/repo/.git/hooks/. What you need to do is to get your git repository to execute a command in trac-admin which updates the tickets with the new commit(-s), namely the command

trac-admin /path/to/folder/wher/you/want/trac/project changeset added <repository-name> <commit-hash>

If you just want this to work for yourself locally, it is quite easy. The script named "post-commit" is executed after every commit, and since you can only commit one commit at the time (right?), you can then just get the commit hash from your log. Add this to post-commit:

#!/bin/sh
commit_hash=$(git log --all -1 --pretty="%H")
trac-admin /path/to/folder/wher/you/want/trac/project changeset added changeset added '(default)' $commit_hash

The repository name is ‘(default)’ by default. This is there to support multiple repository functionality (I haven’t gotten that far yet).

For a more practical case you might want a bare repository that several people have push access to, so that every time someone pushes changes to this repository the tracker gets updated. This complicates things, because there are no hooks that are ran once per commit (to my knowlegde), so the script above would only add the last commit sent in a commit stack. Luckily there is help, and the GitPlugin guys wrote two scripts for the purpose which you can find here. There is one script for pushing changes from all branches, and one script for only letting changes to the master branch be appended to the tickets. Add the one you want into post-receive, which is a hook that is ran after each commit stack has been pushed to the repository.

Now you are all set (if I have remembered all the stages). Try to log into your tracker, create a new ticket, and then try to commit (and push if you chose the second option) and see if your changes appear. You can refer to a given ticket in your commit message with #1, like "This is related to bug #1". In the configuration above we set commit_ticket_update_commands.close to "fixes", which means that if we write "This fixes #1" in our commit message, then ticket #1 will be closed automatically. Obligatory screenshot below.

If you find some missing information then please let me know and I’ll try to update the walkthrough.

Created a flattr account

So, about a month ago I created for myself a flattr account, and today I decided to add my blog as a product that you can flattr if you feel like it. Ideally I would prefer that I could add a clickable icon as a signature to each post, but I do not see any way to do this on WordPress? Does anyone out there have a recommendation on how to best integrate Flattr to a WordPress blog?

Oh, and most of you two readers out there might ask yourself what flattr is? Well, it is a site where you set up an account and add funds. Then you decide how much should go out of your account. Afterwards, you can click on a simple flattr button when you want to donate some money to a product, which could be a video on Vimeo, an article like this one, a linux distribution or whatever other product you can think of really. The monthly amount will automatically be divided between all the things you flattered in a given month. In other words, it is one of the easiest ways I’ve seen yet where you can donate a small amount of money quickly and easily without spending everything on credit card charges. I think this concept has a lot going for it!

Tip of the day: partial git log

This entry is a short one. Do you have a large project and are curious about the changes done in a subfolder only, or perhaps just one file? I noticed today (perhaps this is obvious to anyone but me), that if you use the command "git log <file>" or "git log <folder>", you only get the log from the commits where there was changes in that file or within that folder. This can quite often be useful when you want to track down a version of a particular file…

Latexdiff and git

A lot of scientists have latex as their preferred format for writing articles. A very nice tool when working on latex projects with several people is latexdiff, a perl script that based on two versions of a latex document creates a "diff document" which shows all new (text) additions in blue with curly underline, and all removals in red with a through line.

Latexdiff works well with svn and cvs repositories, where you ask latexdiff to simply compare a given file in two given revisions. However, to my knowledge this has not yet been extended to git. Hence I decided to write a bash script that works on a latex project in a git repository.

In the current form, the script compares the current version against a specified version in the repository. The specified version can be a branch, tag or a revision id. It will include any uncommitted changes as well. It does require that you are currently on a branch. It could probably be extended to work under more general circumstances, but this works fine for me.

One limitation in latexdiff is that it doesn’t work on more than one latex file. On certain big projects (like my PhD thesis), the document is written from several subdocuments using \include{} (or \input{}) statements. I added this to the bash script, so that you can specify several files that have changes you want to include in your pdf document. It actually currently does not fix \input statements since I don’t use them.

You could probably rewrite the script for e.g. mercurial rather easily. If you do, please add a link to it in the comments here.

Update: Instead of just leaving the script in pastebin I added it to gitorious, so it will be easier for you to make changes to it. You can get it with this command:
git clone git://gitorious.org/latexbatchdiff/latexbatchdiff.git
If you have the git plugin for Mercurial installed, you can of course use the same command:
hg clone git://gitorious.org/latexbatchdiff/latexbatchdiff.git

Update 2: Thanks to Paul Anton, I figured out a much better way to solve the problem. The script is now updated so that it must be set up with a command that it can use to get latexdiff in your repository, and then it will work without any need to be in a branch or similar. It does work for me using git, could someone test in mercurial and post here? If you don’t want to check out the repository, you can also have a look at it online:
http://gitorious.org/latexbatchdiff/

Credit also goes to amonakov for giving information on how to set this up for git.

4 million euros to produce france.fr?

Technocrunch had another article today about the price of france.fr. 4 million euros, which is an amazing number for a website. Granted it looks good, but at 4 million it pretty darn should look good as well! I cannot understand any part of developing a web page that can give it such a price tag. Nothing.

Then I read some comments and searched around a bit, and figured that one of their southern neighbours has them beaten by an order of magnitude. italia.it was a whopping 45 million euros to produce according to this site.

It doesn’t stop there though, Romania will be spending 500 million euros in the next three years on their e-portal according to this site. That is a total of 25 euros per person living there, to be compared with their GDP which according to wikipedia is 12 600 USD. That almost makes me sad.

You got worse stories? I would be curious to know about pricetags of similar Norwegian projects. I would not be surprised if some of them were rather ridiculous as well.

Git backup with Dropbox

Today I wanted to write how I have set up my git repositories for backup. At work I have quite limited backed up space, but scratch space which is not backed up I get plenty of. I keep my different projects in git repositories, and I also mirror the svn repositories for the programs I use at CERN (they just recently moved from CVS to SVN).

Just to point out this to users that might get confused: The command && just means that I combine two lines, to save linespace in this blog.. It is basically for me to point out where you have to use a certain command.

First I create the repositories on the server space I have. If you do not know how to do that (git init && git add <file> && git commit), then have a look at a brief tutorial to git before you continue.

Next in each repository I create a remote repository with the name "dropboxbackup", like this:

git remote add dropboxbackup file:///home/myusername/Dropbox/git/nameofgitrepo/

Next I need to initialize the new repository in Dropbox. It is purely backup, so I choose a bare repository:

cd ~/Dropbox/git/nameofgitrepo/ && git init --bare

Now, I can backup my repository with the following command (inside the repository on my server)

git push --mirror dropboxbackup

In order to simplify things, you can add a new alias in git:

cd /path/to/gitrepo/on/server/ && git config alias.backup "push --mirror dropboxbackup"

This way you can run the backup with

git backup

This is getting quite easy, but one thing which is easy to forget is to actually back up. This is where cron come into play. First make sure the cron daemon is running on your system. Next, you write a small script "backupGit.sh" with the following content:

#!/bin/bash

date
cd /path/to/gitrepo/on/server/
pwd
git backup
cd /path/to/gitrepo2/on/server/
pwd
git backup
...

Finally you add the job to cron with the command "crontab -e"

@hourly ID=backupgitrepos nice -n 19 /path/to/script/backupGit.sh >> /path/to/script/backupLog.txt 2>&1

Notice that I used “>>”, which means the file backupLog.txt must exist before the job is ran. You can do this with “touch /path/to/script/backupLog.txt“. This will make sure that every hour your git repositories will be backed up to Dropbox. Have a look in the backupLog.txt file where you can see if everything is going as expected. You can of course still backup manually with "git backup" in addition to the automatic backup whenever you want. For me this means that I will have a copy of the repositories on the server, on all my computers that have installed the dropbox client and on the dropbox server itself. That is actually significantly better than the daily backup they offer at work!

If something was confusing, or you have suggestions for a better setup, let the comments flow!

First meeting with the Nokia Qt SDK

A long time since I have written anything now, so I thought I could write my early experiences with Qt-creator, and development for the Maemo platform. Last weekend I decided I wanted to see how difficult it would be to write a "Hello World" application for the N900, using the development kit available. I would just like to point out from the get-go that my experiences with GUI programming is fairly limited. It extends to one course in Java I had about five years ago at the university. Most of it is long forgotten. What I do with computers is programming and running simulation codes, calculating physics stuff. For this it suffices to just have a script available and input/output files. No need for graphical stuff for most of what I do.

I started by installing the QtCreator and playing around with it. It is really easy to get started with. Create a new project and you are greeted with an example code consisting of an empty .ui file (Qt graphical files in XML format), a header/source for the window and a main.cpp file. Open the UI file and you are greeted with a designer, where you can add/remove buttons, lists, layouts etc, and connect them with signals/slots. I don’t think it gets much easier than this.

The image above shows the welcome screen. The Qt Creator has a nice set of features, with drop down menus of functions inside classes, at least for Qt. This helps a lot when writing your code. Below is an example of how the source code window looks.

And finally a screenshot of how the Qt Designer window looks:

In all screenshots you see the current source code of my hello world application. I called it "shoppinglist", and it basically is a list-able thing that probably wont get far. I just needed a simple concept to play around with. I also tried out Github, so you can find my source here if you are curious.

Initially I installed QtCreator from the Arch repositories, and the Maemo SDK virtual image. I then created a "standard" C++ project. To install your application you then need (from what I understood, there might be easier ways) to run the virtual image in e.g. VirtualBox, then you need to copy your project to this virtual machine (I use Dropbox for this), then you need to log into "scratchbox" inside this virtual machine, and generate/modify the debian packages. Finally you need to build the package and move the *.deb file to your mobile. This takes a while to figure out the first time, let me tell you!

A much easier way I found later on, was to download the "Nokia Qt SDK" instead, which comes complete with a mobile simulator and possibility to build for Maemo (and Symbian S60 in the Windows edition from what I could understand). What you then do is simply open a new project, select Mobile Qt Application and select where to install (let the boxes for Maemo and the simulator be checked). Then you have a one-click compile&run to get your application shown inside a simulator, as shown in the screenshot below.

If you then want to build for your phone, you simply select Maemo instead of simulator in the bottom left, and click build. It then automatically compiles for ARM architecture and creates a *.deb file of your project. This file you can then install on your phone by first getting root access (e.g. with the gainroot script), and then run "dpkg -i <debfile>".

By default the binary is installed in /usr/local/bin/ which isn’t part of the Maemo path. You also need to do some other modifications to the debian files in order to get them properly working, but at least it is a very good starting point. There are some quirks here and there, like how the simulator isn’t entirely accurate all the time, but all in all I am very pleased to see just how much easier this has gotten! Perhaps the joy for people that have seen the hard way is larger, but all in all quite impressive right?

Finally, a screenshot of my application running on my N900: