Python 2.5 and Boost.Python

As Python 2.5 is now unmasked in Gentoo I decided to try it with Boost.Python, more specifically with Python bindings for Paludis. Everything seems to work, but, of course, boost needs to be rebuilt first.

Boost.Python: docstrings in enums

For quite some time I wanted to add docstrings to enums exposed with Boost.Python as my project's API docs seemed incomplete. Let's see how I found the solution eventually:

Enum we want to expose:

// Nice enum!
enum Foo
{
    blah
};

While exposing the enum itself couldn't be any simpler:

BOOST_PYTHON_MODULE(enum_test)
{
    bp::enum_<Foo> enum_foo("Foo");
    enum_foo.value("blah", blah);
    ...

... adding __doc__ is completly another story.

First (silly) attempt:

...
    PyObject_SetAttrString(enum_foo.ptr(), "__doc__",
        PyString_FromString("Nice enum!"));
};

"It must work" I thought, but what I got on import then was far from satisfying:

TypeError: attribute '__doc__' of 'type' objects is not writable

After digging in Boost.Python, Python C API docs and harassing people at #python:

...
    PyTypeObject * pto = reinterpret_cast(enum_foo.ptr());
    pto->tp_doc = "Nice enum!";
};

Thils looked really promising... and what? Nothing at all! Changing tp_doc had no effect whatsoever, and only after some more digging, it turned out it was too late to change it and the right way is:

...
    PyTypeObject * pto = reinterpret_cast(enum_foo.ptr());
    PyDict_SetItemString(pto->tp_dict, "__doc__",
        PyString_FromString("Nice enum!"));
};

Boost.Python and Python C API are fun ;)

Python bindings now in the scm ebuild!

Python bindings are now available in the paludis-scm.ebuild from the Paludis overlay. Just set the python use-flag and reinstall. Comments are welcome, but keep in mind it's far from the final version :]

Additional links: API docs, repository

Python bindings for Paludis #1

I was supposed to give some updates about my SoC project, so here is the much delayed first one.

I have been working on the bindings since the moment I thought about the project and had some code ready even for the initial project proposal. After it had been accepted I was continuing the work with bigger and smaller breaks. Meanwhile I was doing some contributions to Paludis in other areas, learning its internals and getting better at C++.

I believe the first big date for my project was 5 April 2007 when bindings code was initially imported to the Paludis repository (r2881). The second big date is still coming, which will be the release of Paludis 0.26 with the Python bindings included.

The exact status of the bindings is best seen in the repository or in the API docs, which I update every now and then.

Currently I am working on bringing *DepSpec up to date and preparing for yet another Paludis API change. The bigger plan is to finish bindings for all the core classes sometime soon...

UNINSTALL_PROTECT in Paludis #2

Update: Check this for a better performing python version of this hook.

I need to make a follow-up again as the hack described in my previous post is now obsolote as Paludis' trunk has now _override hooks for merger and unmerger actions. They are a little different than all the hooks till now as their effect is determined from their output, more specifically unmerger_unlink_*_override has two options:

• "skip" - skips the action
• "force" - force it (without the usual tests like type or mtime check)

And merger_install_*_override has only the "skip" option. Also no output means the default.

Let me show an example:

/etc/paludis/hooks/unmerger_unlink_file_override/uninstall_protect.hook:

#!/bin/bash

UNINSTALL_PROTECT="/lib64/modules/"

hook_run_unmerger_unlink_file_override() {
    for PROTECT in ${UNINSTALL_PROTECT}; do
        if [[ "${UNLINK_TARGET}" == "${PROTECT}"* ]]; then
            echo "skip"
        fi
    done
}

As easily you can make INSTALL_MASK hook and others...

This and more in forthcoming 0.26.

UNINSTALL_PROTECT in Paludis

Update: Check this for the current bash version of this hook or this for a better performing python one.

This is somehow related to my previous posts about the moduledb as the idea also arose when playing with kernel modules.

When installing modules for a new kernel I noticed that Paludis treated them as any other package and removed the old installs leaving my old kernel without the modules. Nothing really surprising here, just different than the portage behaviour. Wanting to avoid this in the future my first thought was just to add /lib64/modules to the CONFIG_PROTECT, but after a second I realised it wasn't the best idea as I didn't want to review modules' changes when upgrading, I just wanted them to be not removed when uninstalling. Hence that it wasn't long until making an Unmerger Hook came to my mind.

/etc/paludis/hooks/uninstall_protect.hook:

#!/bin/bash

UNINSTALL_PROTECT="/lib64/modules/"

hook_run_unmerger_unlink_file_pre() {
    for PROTECT in ${UNINSTALL_PROTECT}; do
        if [[ "${UNLINK_TARGET}" == "${PROTECT}"* ]]; then
            mv "${UNLINK_TARGET}" "${UNLINK_TARGET}.protect"
        fi
    done
}

hook_run_unmerger_unlink_file_post() {
    if [[ -e "${UNLINK_TARGET}.protect" ]]; then
        mv "${UNLINK_TARGET}.protect" "${UNLINK_TARGET}"
        echo "protected '${UNLINK_TARGET}'"
    fi
}

And two symlinks pointing at it:

/etc/paludis/hooks/unmerger_unlink_file_pre/uninstall_protect.hook -> ../uninstall_protect.hook

/etc/paludis/hooks/unmerger_unlink_file_post/uninstall_protect.hook -> ../uninstall_protect.hook

moduledb in Paludis #2

This is only a short follow-up to my previous post.
Support of dynamic configuration files has been implemented a few days ago and for a start I have made two simple dynamic sets:

kernel-modules-ver.bash (set of kernel-modules with exact versions):

#!/bin/bash

sed -e 's/.*:/* =/' /var/lib/module-rebuild/moduledb

kernel-modules.bash (set of kernel-modules - unversioned):

#!/bin/bash

shopt -s extglob

while read PKG; do
    PKG=${PKG##*:}
    PKG=${PKG%%-scm*([[:digit:]])}
    PKG=${PKG%%-[[:digit:]]*([^-]|-[^[:digit:]])}
    echo "* ${PKG}"
done < /var/lib/module-rebuild/moduledb

And that's really only the beginning...

moduledb in Paludis

Update: Check this for the current solution.

With portage I was quite often using module-rebuild, more or less after every kernel update and thus I thought it would be nice to make something similar for Paludis, especially as it seemed really simple with Paludis' Hooks and Sets.

So I made two simple hooks:

• ${paludis_confdir}/hooks/ebuild_postinst_post/modules_set_add.bash
• ${paludis_confdir}/hooks/ebuild_postrm_post/modules_set_rm.bash

And a simple one-liner to import entries from the moduledb:

sed -e 's/.*:/* =/' /var/lib/module-rebuild/moduledb > ${PALUDIS_CONFDIR}/sets/modules.conf

But then I realized that it's not so perfect:

• Won't work when using Environment != Paludis ( ${PALUDIS_CONFDIR} won't be reliable then )
• To actually rebuild the modules one has to use: paludis --dl-reinstall always --dl-deps-default discard -i modules and hope that the deps are satisfied :]
• Doubles the work of the linux-mod.eclass

So I decided to talk with Ciaran about the above issues. Yet while we were talking he implemented a new paludis option (--dl-reinstall-targets auto|always|never), which affects the way paludis treats targets. Up till now the default "auto" was always used(="never" for sets and "always" for ordinary packages). When it is released the module rebuild will be simply done with:

paludis --dl-reinstall-targets always -i modules and there won't be any risk of missing deps.

Going further ciaran thought about allowing bash scripts in sets/, in short paludis runs a foo.bash script from sets/ and its output is interpreted as foo set. The moment it is implemented (probably today) and released we can forget about the two hooks above and make a modules.bash similar to the import one-liner.

So don't forget to update Paludis when it's out ;]

My brand new blog and Google Summer of Code

Welcome to my brand new blog!

Thanks to seredipity setting it up was quite easy :] I've been thinking about making one for quite some time and today I have eventually decided to do so. Why today? Well, GSoC!
My proposal was accepted as one of the Gentoo projects. In short, my task is to make python bindings for Paludis, the Other Package Mangler for Gentoo. If you want more details, check out my application and, if you are even more desperate, code, which I have already written. Also ciaran was nice enough to write a few words about it, but don't forget to ignore the rubbish about Python :]
So... I am planing to blog about progress of my project, some Gentoo stuff, and... time will tell.

P.S. 1) My name is Piotr Jaroszyński (hence the domain), I live in Warsaw and am first year Computer Science student at Warsaw University.
P.S. 2) I hope my pathetic 256 kbit/s upload will manage ;]