Python stack in GDB

I’m sure everyone already knows about this, but it’s such a nice feature I’ll post it anyway.

There’s a set of macros for gdb, described in a comment on this page, that will let you attach to a running python program using gdb and inspect its python call stack and python objects using the familiar interface of gdb.   I’m a complete stranger to python and couldn’t figure out how to enable the python debugger, and it would get me lost even if I managed to enable it. Additionally I was trying to find out when and why a python program uses a particular syscall and I’m not sure the python debugger can help with this.  For the record that python program blocks all signals so I couldn’t just send it a signal and have it print the stack.

I’m wondering if you can do the same thing with Java, and who’ll be the first to implement the gdb macros.  I’ve not coded java for years but it makes me want to have a look at it again considering there’s source code for it now (I just wish I had the time). How about swi-prolog?

Practical note: For this to work you will need to rebuild python with debug information in. If you’re on Gentoo, whose default package manager uses python, and if you still have python2.4 installed, if you screw up your python2.5 installation, you can revive emerge by running it implicitly with python2.4 (python2.4 /usr/bin/emerge blah blah).  To rebuild python with custom options, edit /usr/portage/dev-lang/python/python-2.5.2-r8.ebuild to add –with-pydebug, and run ebuild /usr/portage/dev-lang/python/python-2.5.2-r8.ebuild digest unpack, then edit /var/tmp/portage/dev-lang/python-2.5.2-r8/work/Python-2.5.2/Objects/unicodeobject.c to remove the assert on line 372, which seems to be a typo, and then ebuild /usr/portage/dev-lang/python/python-2.5.2-r8.ebuild compile install qmerge to let it finish.  You may need to re-emerge some of the packages that have installed into /usr/lib/python2.5/site-packages) for your program to work again.

Get kiva.org credit from MTV

MTV gives away $25 gift certificates for lending on kiva.org (you’ll need to register)

Public transport season is on

All the times it snowed this winter it first looked like it was going to stop me from biking through the city but was followed by a soon change in weather and the snow melted before I actually needed to go anywhere.  Today I’ll need to make a small trip and the streets are covered with enough snow that melting it in the time left until I’m leaving would require an amount of energy to radiate that would be dangerous to forms of life, so I think it’s the end of the biking season finally.

Accelerating in my pocket

I started poking at the SMedia Glamo chip in the GTA02 this week. First I played with the Linux framebuffer driver and later with decoding MPEG in hardware, and now I have some code ready. I was challenged by messages like this on the Openmoko lists. Contrary to the opinion spreading accross these messages, we’re not doomed and we still have a graphics accelerator in a phone (which is coolness on its own). And it’s a quite hackable one.

I first had a look at libglamo code – a small library written some time ago by Chia-I Wu (olv) and Harald Welte (laf0rge) for accessing some of the Glamo’s submodules (engines). I asked the authors if I could use their code and release it under GPL and they liked the idea, so I stitched together libglamo and mplayer and added the necessary glue drivers. This wasn’t all straight forward because mplayer isn’t really prepared for doing decoding in hardware, even though some support was present. Today I uploaded my mplayer git tree here – see below what it can and cannot do. There’s lots more that can be improved but the basic stuff is there and seems to work. To clone, do this:

cg-clone git://repo.or.cz/mplayer/glamo.git

The Glamo fact sheet claims it can do MPEG-4 and H-263 encoding/decoding at 352×288, 30fps max and 640×480 at 12fps max. Since it also does all the scaling/rotation in hardware, I hoped I would be able to play a 352×288 video scaled to 640×480 at full frame-rate but this doesn’t seem to be the case. The decoding is pretty fast but the scaling takes a while and rotation adds another bit of overhead. That said, even if mplayer is not keeping up with the video’s frame-rate it still shows 0.0% CPU usage in top. There are still many obvious optimisations that can be done (and some less obvious that I don’t know about not being much into graphics). Usage considerations:

• Pass “-vo glamo” to use the glamo driver. The driver should probably be a VIDIX subdriver in mplayer’s source but that would take much more work as VIDIX is very incomplete now, so glamo is a separate output driver (in particular vidix seems to support only “BES” (backend scaler?) type of hw acceleration, which the Glamo also does, but it does much more too). Like vidix, it requires root access to run (we should move the driver to the kernel once there exists a kernel API for video decoders – or maybe to X).
• It only supports MPEG-4 videos, so you should recode if you want to watch something on the phone without using much CPU. H-263 would probably only require some trivial changes in the code. For completeness – MPEG-4 is not backwards compatible with MPEG1 or 2, it’s a separate codec. It’s the one used by most digital cameras and it can be converted to/from with Fabrice Bellard’s ffmpeg. A deblocking filter is supported by the Glamo but the driver doesn’t yet support it. For other codecs, “-vo glamo” will try to help in converting the decoded frames from YUV to RGB (untested), which is normally the last step of decoding.
• The “glamo” driver can take various parameters. Add “:rotate=90″ to rotate (or 180 or 270) – the MPEG engine doesn’t know about the xrandr rotation and they won’t work together. Add “:nosleep” to avoid sleeping in mplayer – this yields slightly better FPS but takes up all your CPU, spinning.
• Supports the “xover” output driver, pass “-vo xover:glamo” to use that (not very useful with a window manager that makes all windows full-screen anyway).
• Only works with the 2.6.22.5 Openmoko kernels. There were some changes in openmoko 2.6.24 patches that disabled access to the MPEG engine but since we don’t have a bisectable git tree I can’t be bothered. UPDATE: A 2.6.24 patch here – note that it can eat your files, no responsibility assumed. I guess it can also be accounted for in mplayer, will check. My rant about lack of changes history in git is still valid – while I loved the switch to git, the SVN was being maintained better in this regard.
• In the mplayer git tree linked above I enabled anonymous unmoderated push access so improvements are welcome and easy to get in.

With respect to the linux framebuffer poking, I wanted to see how much of the text console rendering can be moved to the hardware side and it seems the hw is not lacking anything (scrolling, filling rectagles, cursor) compared to the other accelerated video cards, and even the code already exists in Dodji Seketeli’s Xglamo. I’m sure sooner or later we’ll have it implemented in the kernel too. For now I got the framebuffer to use hardware cursor drawing (alas still with issues).

Bricked! lol

Somewhat related to the Phoenix probe landing, I found in the Viking mission page on wikipedia (the exams are here again and I’m looking up things on WP and then getting stuck reading completely unrelated stuff and consequently failing exams) an amazing bit of information. The mission started in 1975 when it sent to Mars two NASA rockets carrying four spacecraft, each having on-board a computer based on the RCU1802 chip (that was a legitimate computer at that time). All four vessels successfully carried out their missions but each one failed years later in a different way. Three computers were shut down in appropriate ways worth a space travel (physical damage) but the last operating one has this failure reason: Human error during software update.  Sounds so contemporary.

It’s amazing that a board that left Earth in 1975 could be updated from 100,000,000 km away (some vendors still don’t get it about updates). Even more amazing is that the discussion of whether (and how) to protect software from the user is still not resolved. FIC GTA phones evolve a pattern of writable and read-only memories to become “un-brickable”. I’m sure that’s partially because it becomes less clear who is a user and who is the developer (like in a NASA mission). It’s clear that nobody wants their mission to end this way, “a lorry ran over my phone” somehow sounds much better.

Gazpacho a lo guiri

Background: I try lots of new things when I make my food and while most of my experiments fail miserably, there are cases that come out well enough that I even repeat them, so I thought I can share (open-source) one of these results, and this is an attempt. But, open-sourcing food, strangely, isn’t so easy because the “code-base” is very ugly – everything is an undocumented hack (or “spaghetti code”), and needs to be documented. My optimisation flags are always set for minimising the number of dishes to wash and ingredients cost.

Gazpacho: gazpacho is a Spanish dish (or drink) originating from Andalusia. It’s made of mainly vegetables, is almost liquid, is consumed cold and doesn’t involve cooking. It’s eaten in the summer only, especially on hot days. (At higher latitudes than Spain, I found the added practical argument is that vegetables are about 3 times cheaper in the summer).

There are a zillion types of gazpacho and some Spanish are very religious about preparing it, especially those who make cookbooks. Every region has its own type, but there’s also the generic type that you can buy in supermarkets or in McDonald’s.

Now, I’m not Spanish and I allow myself to break some of the rules. If you’re Spanish, stop reading here because you’ll find that I’m committing various terrible crimes against the mediterranean cuisine.

The recipe: today I completed a quest for all the ingredients and made a gazpacho again and it turned out eatable again, so it must not be extremely difficult, here’s the list.

• 0.5 to 1kg of tomatoes (canned tomatoes will also do, even a box of juice – here’s where I commit the first crime).
• 2 red peppers/paprikas, optionally add one green.
• 2 or so slices of bread.
• one half bulb of garlic (maybe less).
• a medium-size onion.
• a cucumber.
• 1tbs or so of salt, some pepper (or none).
• half a glass of oil (here’s my second crime: use any oil – it really really doesn’t matter that much. It appears that if you’re Spanish a single drop of oil that is not the absolute highest quality immediately spoils your dinner. You will never ever see or hear the word oil (aceite) go alone when you’re in Spain – it is in 100% cases accompanied by the words virgen and extra, as in aceite de oliva virgen extra – it might equally well just be a single word in the vocabulary because it always appears together, I don’t think anyone is even able to pronounce aceite alone).
• 4tbs or so of vinagre and half glass of water.

Place the bread in a plate with water and let it dissolve a bit. Cut all the vegetables into pieces of sizes that will make your electric blender happy. The peppers and tomatoes are fine as they are (another crime!) – in a real gazpacho recipe they tell you to peel them and remove the seeds, but the seeds are the best part, they’ll get blended anyway and they’ll just make the texture nicer, and peeling is just too much work. Blend the bread, vegetables, and water until liquid. Then throw in the oil, vinagre and salt and mix again until the oil can’t be seen.

The colour is between orange and pink and is most influenced by the red peppers. The taste is most affected by the salt and vinagre and you need to adjust their amounts but it will probably be a lot of vinagre and a lot of salt. Too much onion or garlic makes the gazpacho spicy but at some point the smell is too strong.

When it’s done, just store in a fridge and serve cold optionally with pieces of toasted bread.

Unscientific GPS note

Last week I charged the different batteries and took a GTA01 Neo, a GTA02 Neo and a Nokia N810 with me to enable their GPSes on my way home from school. Then I saved the traces they logged and loaded into JOSM to have a look (GTA01, GTA02, N810 – gpx files converted using gpsbabel from nmea)

The devices made respectively 11.28km, 12.12km and 11.07km routes (sitting in the same bag the whole time).

All in all I like the GTA01 accuracy the most although all three sometimes have horrible errors. They all three have accuracy about near the bottom line of usability for OSM mapping for a city, so if you get a GPS with that in mind, it may be slightly disappointing. All three are quite good at keeping the fix while indoor but everytime there’s not enough real input available they will invent their own rather than admit (if you had physics experiments at high-school and had to prove theories that way, you know how this works), resulting in run-offs to alternative realities – especially the N810 likes to make virtual trips. They all three apparently do advanced extrapolation and most of the time get things right, but the GTA01 GPS (the hammerhead) very notably assumes in all the calculations that the vehicle in which you move has a certain inertia and treats tight turns as errors. I’m on a bike most of the time and can turn very quickly and it feels as if the firmware was made for a car (SpeedEvil thinks rather a supertanker).

It’s suprising how well they all three can determine the direction in which they’re pointing even when not moving (the GTAs more so). The firmwares seem to rely on that more than on the actual position data sometimes. This results in a funny effect that the errors they make are very consistent even if very big – once the GPS thinks it’s on the other side of a river from you (or worse in the middle), it will stay there as long as you keep going along the river.

I’m curious to see what improvement the galileo system brings over GPS.

UPDATE: I was curious about the precision with which the altitude is reported, which can’t be seen in JOSM.  First I found that the $GPGGA sentences on my GTA01 have always 000.0 in the elevation field, but the field before it (normally containing HDOP) has a value that kind of makes sense as an altitude, so I swapped the two fields (HDOP value should be < 20.0 I believe?).  Then I loaded the data into gnuplot to generate this chart:

The horizontal axis has longitude and vertical the elevation in metres above mean sea level.  Err, sure?  I might have screwed something up but I checked everything twice.  Except the GTA01 which might be a different value completely – but the is some correlation.  I’m not sure which one to trust now.

Panoramic photos revisited

Every some time there’s a place that would look great on a panoramic picture but I only have a plain touristic camera with me and decide to take a series of photos in all directions from one point with a plan to later glue them together to get something real. Then I forget it completely or sometimes I load Gimp and glue the pictures together. This can take an hour or two and gets boring, but the result is often ok (digital example, analog camera example). This time I decided to try to get my PC to do the gluing for me and use one of the tools that appeared in gentoo’s portage tree. This didn’t go faster than I would have done it manually (perhaps five x longer) but knowing about all the quirks, it may actually go faster next time, and the result is comparable with manual stitching. So let me just write down the things I wish I had known when I started. The package that is now in most distro’s repositories and that I used, is libpano12 and various tools associated with it.

First part is selecting the individual pictures and telling the computer how they are oriented in relation to each other so it knows how to glue them. Hugin (a GUI frontend for libpano) lets you select the shots and input the data necessary for libpano to make sense of the individual pictures, which is a list of common points on the overlapping parts of the photos. This part quite intuitive. Remember to build the latest hugin and latest wxGTK or hugin will segfault. Before loading pictures into hugin rotate them in Gimp to be at least more or less straight (if they aren’t). Have 2GB of free disc space. Choose one of the rectangular projection types because the fancy ones are not supported by the other tools we’ll need to use. The picking of common points can be done automatically by autopano-sift but this has mono and other heavy packages as dependencies so I avoided it. When you’re done placing the points and playing with all the other parameters in hugin, hugin will want to generate a script for the stitching program that will do the heavy work. Hugin knows about two such programs: PTstitcher (part of panotools) which sucks because it’s closed-source and only works on one arch. The second one is nona which sucks because it doesn’t support most of the format, projections, blending, auto adjusting. So we will choose “multiple TIFF” as the output format, set all the other parameters, save the project, tell hugin to generate the script to a text file and quit hugin. Now we have one other program to do the stitching: PTmender, an opensource replacement for PTstitcher, also part of panotools – this one supports the formats we want and some automatic colour balance / exposure correction but if you choose any of the plain bitmap output formats, it won’t blend them nicely together because it’s not supported yet. If you in turn want a format with the individual pieces on separate layers (XCF not supported yet, so PSD) to do the merging in Gimp, PTmender will segfault in some doubly-linked-list code. So we choose the “multiple TIFF” output format and get TIFFs that are ready to merge except they don’t have any transparency mask set. We could now use Gimp’s “alpha to selection” and “feather selection”, but there’s a program, enblend, that will do just this merging, and does it really well (using multi-resolution splines). It only operates on TIFFs, and seems to get all the parameters right if you don’t give any.

You may need to repeat some part of the process if you see something really bad in the enblend output, and if not then you just need to load it in Gimp and rotate / crop / scale the picture. To load TIFFs in Gimp you may need to rebuild with USE=”tiff” if you’re on Gentoo.

Trip

So I went to Brazil last month but had no time to put any pictures online, now I uploaded them here. Also uploaded some pictures from a trip to Spain that was just before that.

Brazil cities reminded me a lot of Peru, which was the only place I had seen in America (this comparison must seem awfully ignorant to anyone who lives in some place between Brazil and Peru). We spent one week in Ceará region seeking out best places for paragliding. One of the spots was the launch pad near Nossa Senhora Imaculada sanctuary near Quixada where “Sol” group (Brazil) took off last year and set the current world record in straight distance paraglider flight landing over 460km away. (Obviously this was a different season and incomparable weather conditions.)  I made an attempt to adapt my Neo1973 Linux phone to dub as a variometer using the altitude data from built-in GPS.  Impressively the measures are somewhere on the edge of being accurate enough for that purpose, but time resolution is way too low (normal variometers use air pressure changes rather than GPS).  The speaker is loud enough to emit the familiar beeping of a variometer (so good enough for showing off even if inaccurate).  The GTA02 should be much better with its 3D accelerometers, but I didn’t have time to play with it yet.

The second week the group split and I went Bossa ‘08 conference that was in a fantastic setting and from where I brought home a collection of five geeky t-shirts.

I could’ve written a novel with all the keystrokes

In other words if you printed all the vim commands I typed, on Earth’s equator in 15pt font, you would get a very repetitive string printed on Earth’s equator line (which could no way be seen from outer space).

$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}' | sort -rn | head
2673 vim
1475 cd
694 screen
632 make
475 man
329 ls
261 grep
231 cg-diff
161 aoss
149 cg-patch

The numbers sum to > 1000 because I use HISTSIZE=”10000″ and HISTCONTROL=”ignoredups” in my bashrc.