The Shoes of the Fisherman's Wife Are Some Jive-Ass Slippers

Fri, 09 Dec 2005

White Christmas

Back in Ft Collins for the last part of my flying visit. It only took 15 minutes for a guy to arrive with a portable battery and jumper leads and get my car started so I was out of California in no time. I don't remember having the NRMA in Canberra turn up anytime sooner than an hour after calling. The plane out of San Jose was over an hour late, so I didn't get to the hotel until after midnight. I was a bit freaked out having to drive 60 miles in the snow and ice but it turned out OK. There were only a couple of places where there was snow covering the road and other drivers just zoomed along at 75mph anyway.

My boss neglected to mention that the temperature here would be below zero, and that's Fahrenheit not Celsius! Apparently people live quite happily in this weather. I am sure that I get a small dose of altitude sickness when first arriving as I get headaches and nausea on the plane in which lasts for half a day or so.

Wed, 07 Dec 2005

Do You Know the Way to San Jose

I'm in San Jose doing a conference and flying visit to see various people. To begin it was quite an exciting trip as I missed my international connection to San Francisco in Sydney but managed to get on the flight to Los Angeles an hour later. That turned out to be quite lucky as the San Francisco flight ended up being cancelled due to a malfunctioning air conditioner. Who knew that the inside of airplanes could get so hot?

The conference is interesting and well organised with lots of technical talks and people to meet and talk to. I have met a bunch of HP people who profess to be interested in WBEM - HP is well represented here.

To Ft Collins tomorrow after figuring out what to do with my rental car after discovering I left the lights on all last night and today. Luckily there is a rental car return at the hotel, so I may be able to get away with dumping the car here and catching a taxi to the airport.

Mon, 14 Nov 2005

Samba Network Interfaces

There was a bit of chatter about using Samba with multiple network interfaces on the main mailing list recently, so I decided to write an article about it: Multiple Network Interfaces with Samba. It was also a bit of an excuse to fool around with Docbook. It's quite neat, although quite hard to get to grips with. I ended up using xmlto instead of any of the more complicated solutions available.

Thu, 03 Nov 2005

Tool of the Month

After installing Breezy Badger on a new machine the other day, I downloaded some ISOs and was suprised to discover that ISO files are now supported in Gnome's archive manager application, File Roller. Just right click on an ISO file in Nautilus and select "Open in archive manager". Very cool, although it has apparently been supported since May 2004. (-:

Wed, 02 Nov 2005

External Antenna

Woot - finally after years of procrastination we have an external UHF/VHF antenna. I am hoping that when the sun goes down, the traditional time when the television reception starts to get crappy, we will not have to worry about wiggling those stupid rabbit's ears around. I haven't tried out the DVB-T reception. That should improve as well.

Tue, 01 Nov 2005

New MythTV Box

My wife convinced me that a new mythtv box would be the way to go, as my old desktop is slowly dying, and my attempts at a small, separate front end didn't work so well (hello Zensonic). I withdrew a wad of cash on Saturday morning and headed down to the bi-weekly computer fair. I ended up buying the following:

• ASUS A8N-E (socket 939, dual-channel DDR, PCI-e)
• AMD64 3000+
• 1GB dual channel DDR400
• Gigabyte 6600GT PCI-e (DVI out, funky heat pipe cooling)
• 1x 120GB PATA disk for OS
• 1x 250GB SATA disk for recordings (will add another 2x or 3x of these later)

I moved the DVD burner and the two DVICO FusionHDTV-Lite cards across from the old machine.

I also splurged on a Sonata II 450W quiet case and power supply which, I have to say, is pretty neat. For the extra $100 or so over the price of a regular case, you get a quiet power supply which is good in itself, and also funky things like drive rails and a backplane that holds the PCI cards without any metal having to be bent. That was a strange experience given that I usually buy dodgy cases.

I'm sure the last time I built a PC that installing the operating system was the hardest bit. This time I popped in a Ubuntu CD (after swapping around the IDE cable - I installed it upside down) and Linux practically installed itself while I was cleaning up the boxes and ripped open plastic bags. Didn't get a chance to install MythTV just yet. Maybe later on in the week.

Mon, 31 Oct 2005

OpenWRT and OpenVPN

Duane Groth of CAcert has nice blog entry which links to his guide on Properly securing wireless networks on the cheap. He uses a combination of OpenWRT and OpenVPN.

I just discovered that two of my neighbours have set up wireless networks in my area (one unencrypted - woot!) so I have been meaning to get around to getting serious about security, which means some kind of VPN solution rather than relying on plain old WEP encryption.

Mon, 17 Oct 2005

On the Origin of Yak Shaving

The Jargon File definition for yak shaving is pretty boring. Here's a better description of this common activity. From some MIT guy:

"I was working on my thesis and realized I needed a reference. I'd seen a post on comp.arch recently that cited a paper, so I fired up gnus. While I was searching the for the post, I came across another post whose MIME encoding screwed up my ancient version of gnus, so I stopped and downloaded the latest version of gnus.

"Unfortunately, the new version of gnus didn't work with emacs 18, so I downloaded and built emacs 20. Of course, then I had to install updated versions of a half-dozen other packages to keep other users from hurting me. When I finally tried to use the new gnus, it kept crapping out on my old configuration. And that's why I'm deep in the gnus info pages and my .emacs file -- and yet it's all part of working on my thesis."

Go shave a yak today!

Wed, 12 Oct 2005

No Electricity

I was planning to work from home today but having not taken much notice of the recent letters from the electricity company was a bit surprised when everything powered down early this morning. I think I need a UPS for the wireless access point.

Actually a charged up car battery, a solar panel and a transformer to step down 12+V to something that my various devices would like could keep me going all day. Also, thank goodness for gas-powered stoves and matches. (-:

Fri, 07 Oct 2005

MONT 24-Hour

Went out yesterday to help set up our camp for The Mont 24-Hour which is a 24 hour continuous montain bike event. We have a pretty primo spot just off the last bit of track before the start/finish line, but away from the generators and parking area.

Last year was much fun, although I could only manage three laps of the 18km course. I think I can manage four this year, but we may not have enough time depending on how the day pans out. Actually this year should be easier, as we will have a full team of six (one last minute scratching last year due to illness) and slower riders so there may be a rest period of up to six hours between laps, instead of four.

There are forecasts of rain for the next three days, and the course is already wet (frogs chirping and running water alongside the track). I am skeptical in general of forecasts of rain due to the recent drought in Canberra, but we dug drainage ditches around the tents anyway.

Tue, 04 Oct 2005

Neat format for software release notes (NEWS)

mbp has a neat format for formatting the release notes for small to medium sized software projects. It's being used for distcc and Bazaar-NG. The idea is that changes in the piece of software are sorted into maybe half a dozen different categories with a short sentence or two, with attribution, is written about the change. The style is very much in the GNU ChangeLog format.

Here's the news so far for my PyWBEM project for the upcoming release:

pywbem-0.4  "Funky release name"  RELEASE-DATE

  BUG FIXES:

    * Correctly calculate value of Content-Length HTTP header to include
      opening XML stanza.  Reported by Szalai Ferenc <szferi@niif.hu>.

  ENHANCEMENTS:

    * Support for marshaling and unmarshaling CIM dates object into
      Python datetime objects.  Written by Szalai Ferenc <szferi@niif.hu>.

    * Experimental module for making asynchronous WBEM calls with
      PyWBEM in Twisted Python.

    * Add parameter type checking for object location classes.

  API CHANGES:

    * Change association provider API functions to take a fixed
      parameter for the named object instead of a keyword argument.
      This breaks backward compatibility.

    * Remove the CIMLocalNamespacePath class and replaced by a Python
      string.

  PORTABILITY:
 
    * Don't use UserDict.DictMixin base class as it only exists in
      Python 2.3 and higher.

  TESTS:

    * Add tests for parameter type checking for object location
      classes.

    * Add tests for string representation of object location classes.

The things I like about this format are:

• It neatly separates bug fixes from enhancements and improvements. Splitting up code changes into different categories like this gives a good overview of how the software project is going and the flavour of the release.
• It's easy to credit contributors of bug reports, patches and the implementation of new features. Attribution is such an important part of open source development, and failing to give credit for a contribution may discourage someone from submitting another patch or fixing another bug for you.
• You can add whatever categories you like as the format is fairly unstructured giving some freedom in what is written, but structured enough to write incrementally over a period of months as work on the project continues.

Suggestions for a funky release name for PyWBEM 0.4 will be gratefully received!

Tue, 27 Sep 2005

More Unix Secrets of Billionaires

mbp mentions a neat notification trick for reducing turnaround time when running jobs that take more than a couple of tens of seconds. Here's my modified version. It plays a different sound based on the success or failure of the command, and also preserves the exit code.

#!/bin/sh

"$@"
rc=$?

if [ $rc = 0 ]; then
	snd=/usr/share/games/neverdata/snd/success.ogg
else
	snd=/usr/share/games/neverdata/snd/over.ogg
fi

ogg123 -q $snd
exit $rc

I used to have a script that played the sound of some glass breaking when the job failed, but the noise was too jarring and would often make me jump when it happened unexpectedly. The sounds from neverball are a bit less frightening.

(From the neverball FAQ: "My computer does not meet the stated minimum system requirements, and yet I insist on trying to play it anyway and blaming you, rlk, for writing a crappy game. WTF?").

Video Editing With Linux

I watched a friend use the TMPGEnc MPEG Editor to remove commercials from a MPEG file recorded on his Topfield the other day and was duly impressed. I was curious whether this can be done under Linux or not, as I tried and failed to get the TMPGEnc editor running under Wine.

As usual, there are many half-assed programs on sourceforge that don't work very well or haven't been updated in ages, but GOPchop sounded like it would do the trick. It's not available in Debian, and the prebuilt deb on the site had various dependency problems, but it wasn't too hard to build by hand. I removed some leading and trailing junk from some TV I recorded last night in a couple of minutes. The only slow bit was waiting for the MPEG file to be loaded and scanned, and then writing out the MPEG file containing the cuts.

I guess this isn't really "editing" in the professional sense of the word, but removing commercials and trimming MPEG2 files is really all I need to do at the moment. Next step of my PVR is to have a slightly better interface than running dvbstream in an xterm and hitting Ctrl-C when the program has finished.

Fri, 23 Sep 2005

Useful find scripts

Once upon a time I had a funky script that would run the find(1) command on all the C and C++ files in the current directory. It was called findch. Now you might think that this was a bit pointless, but it is actually a big timesaver, especially since searching all C files for a particular pattern is usually part of a larger task so a couple of seconds saved reduces the probability of your flow being interrupted. Here's the latest version of my findch script:

#!/bin/sh
[ -n "$1" ] && base="$1" && shift || base="."
find $base -name '*.[ch]' -or -name '*.cpp' -or -name '*.hpp' "$@"

Now that newer version control programs (for example Subversion and bzr) store file contents in their metadata directories, a naive find . -type f | xargs grep foo command will generate a large number of false matches against the metadata files. Here's a similar script that runs find but ignores all the VC metadata:

#!/bin/sh
[ -n "$1" ] && base="$1" && shift || base="."
find $base \( -name .svn -o -name CVS -name .bzr \) -prune -o -print "$@"

I can never remember the syntax for the -prune option.

Sun, 18 Sep 2005

Dual DVB-T Cards Working

I've finally figured out why I haven't been able to get both tuners streaming mpeg2 simultaneously. Predictably enough, it wasn't a bug in the FusionHDTV drivers, but rather in my script to tune and transmit a DVB transport stream using tzap and dvbstream. Running two copies of the tuner script use about 22% of my 800MHz CPU which leaves plenty of cycles available for running a media server and other things.

I can think of a couple of good ideas for implementing a live tv type of arrangement on the Z400 using some tricks in the media server. I still need to fix up the existing Python media server code and upload it somewhere, then to get video files playing.

Thu, 08 Sep 2005

On the Subject of Jive-Ass Slippers

From everything2:

"The Shoes of the Fisherman's Wife Are Some Jive Ass Slippers" is an amazing composition by Charles Mingus for his big band. The orchestration was done by Sy Johnson. Mingus has a penchant for creative titles; some other great titles are "Slop", "Pithecanthropus Erectus", "Gunslinging Bird", "E's Flat, Ah's Flat Too", and "New Now Know How". Of course the main point is that "Shoes of the Fisherman's Wife" is a brilliant composition, as Mingus was a brilliant composer.

My favourite recording of the abovementioned song is by two blues/folk guitarists, Stefan Grossman and John Renbourne, from their album Stefan Grossman & John Renbourn Live In Concert.

Now I could have called my blog "Prepare me a ship of which the half-rotten timbers shall be painted black, let the sails be in rags, and the sailors infirm and sickly", but that's just silly.

Another Twisted Tutorial

I found another tutorial on Twisted Python. It is slides from an OSCON 2003 talk by Itamar Shtull-Trauring, one of the Twisted authors.

Tue, 06 Sep 2005

Media Server Playing MP3s

Woot! After yet another hiatus I have been doing some more poking around on the Python UPnP Media Server. Mainly this was because of some particularly bad analog reception the other day resulting in unwatchable stuff on the tivo. I do actually have two DVB-T PCI cards sitting in my desktop machine but without a Linux media server there is no way to watch any of the recorded data without the family huddling around a computer screen or laptop.

Anyway, in my previous coding session I was able to construct replies sufficient to browse a directory heirachy on the Z400 but it wouldn't actually play any media. It turns out that the Z400 refuses to play any media files that are present at the very top level of the directory heirachy. I made a dummy folder and then stuck a DIDL object.item.audioItem.musicTrack in it, and it works! I am now listening to a really old mp3 podcast of an interview with Severed Heads that was played on Radio National months ago through my own media server.

In the bad news department, the hard drive on my laptop was making some odd noises this morning when restoring a suspended VMware image. There were a whole bunch of the dreaded UncorrectableError messages in the log file. The hard disk is probably going to crash sometime soon. I'd better check in my changes before it does.

Sun, 04 Sep 2005

Google Pedometer

Here's a map of my favourite run around Black Mountain in Canberra using the very cool Google Pedometer. I had a feeling it was about 15km. Google says it's 15.3km so I was pretty close. Unfortunately satellite maps don't give much of an indication of height as the bike path around the north side of the mountain is about a 3km climb.

Mon, 15 Aug 2005

The TSA can go inspect my smelly underwear

Oh wait, they did. Back in Canberra finally. Woohoo!

Sat, 13 Aug 2005

Fort Collins

I have been in Fort Collins all this week for work and it has been quite pleasant. My last trip was in December at the start of winter. There was snow and ice everywhere, although there was a bit of a warm spell for that week so the temperature range was't that different for a Canberra winter (0-12 degrees C). This time is was 20-30 degrees C with a couple of short showers towards the end of the week.

Summer looks like lots of fun over there. On the weekend I saw many people riding their pushbikes and motorbikes (often with no helmets!!) around as well as joggers and dog-walkers out enjoying the sunny weather. I went out with Randy for a very short hike and some rock climbing on the last day I was there. He also took me out for free (woohoo!) samples at the New Belgium Brewery which was pretty good really. I don't think I'm much a fan of wheat beer or Belgian ale though. The ale is definitely an acquired taste. Fort Collins has a high density of microbreweries and regular breweries. There is a giant sized Budweiser plant on the side of the highway on the drive up from Denver if you like that sort of thing.

The Fort Collins office is so very quiet in comparison to Canberra where there are mobile phones going off in your face every 10 minutes and sales guys hang their arm over the side of your cube to have a loud conversation with someone else. I guess it's a difference between an office that does primarily sales and technical support versus an office that does engineering.

Thu, 11 Aug 2005

CIFS 2005

Last week I was at the CIFS 2005 conference, now part of the CIFS 2005 conference, now part of the Developer Solutions Conference & Showcase run by SNIA. As per the the last couple of years, the technical content has been slowly decreasing in quantity and quality, as have the count of companies attending and the number of engineers they have been sending.

I attended some of the Developer Solutions talks about WBEM which were a bit waffly. I don't think WBEM has quite hit the mainstream enough for people to present technical talks about it just yet. iSCSI is apparently on teh spoke at the moment. The talks were apparently well attended and interesting. There were also iSCSI startup-type companies, something which has been lacking at CIFS for quite a while now.

Still, it was great to meet some of the new Samba Team members and put a face and personality to some of the nicks on IRC. I also picked up some cool tips on general Linux stuff and learned all about quilt (googling for quilt doesn't bring up any computer references on the first page at all - WTF?) which was lots of fun.

There are some photos from the conference on samba.org.

Wed, 03 Aug 2005

Man of the match award...

...goes to Andrew Bartlett for finding out about ssh master connections working with the dodgy proxy we seem to be behind at the moment. Master connections are somewhat like fsh in that they keep one connection open all the time, and subsequent ones are multiplexed over it. I never could get fsh working reliably last I tried but master connections are working well.

From the changelog for OpenSSH 3.9:

* Implement session multiplexing: a single ssh(1) connection can now carry 
  multiple login/command/file transfer sessions. Refer to the "ControlMaster" 
  and "ControlPath" options in ssh_config(5) for more information

Add the following to your $HOME/.ssh/config file:

host svn.samba.org
	ControlPath ~/.ssh/master.svn.samba.org

and then run ssh -M svn.samba.org to start the first connection. tridge recommends running watch true in the master connection window to check that the connection is still active if you are worried about that sort of thing.

Tue, 26 Jul 2005

Free Music!

I was searching around the web for information on a particular piece of music I was listening to (it was Herr Gott, nun schleuss den Himmel auf BWV 617 by J.S Bach) and came across a Free Sheetmusic Library. This site makes available PDF files of public domain music in editions that are in the public domain.

I had always been curious as to why I need to pay $A30 and above for music in which subsists copyright that has long since expired, notwithstanding the fact that the music was composed well before today's modern copyright laws even existed. When buying Bach, apart from the costs of printing, transportation, etc, you are also paying for editorial additions (fingering, ornamentation and performance markings). This means the publisher can slap on another 75 years of copyright. Slighly odd is the fact that Urtext (original text) editions pride themselves on the lack of additions to the composer's manuscript. In Australia, Peters Urtext editions are usually more expensive than other editions, but this could be explained by a brand premium or the fact that the pieces of paper are printed overseas and imported.

Is the publisher of an Urtext edition entitled to claim copyright on the publication? Or does copyright only exist on the collection of works and not individual pieces? I think this is quite an interesting question.

The French word patron means manager or boss, and patronage of the crown or a rich nobleman was one "business model" for musicians of that time. I don't think Bach had a stream of royalties available from the publishing and sale of his works. Please note the interesting comparison to open source and free software here.

See the Wikipedia entry on copyright for some light reading, especially the section on unusual copyright grants.

Tue, 19 Jul 2005

Bug du Jour

The IRIX C compiler was found to give the following error (Samba bugzilla 2893):

cc-1184 cc: WARNING File = utils/net_rpc_printer.c, Line = 1356
  "=" is used where where "==" may have been intended.

                if (!W_ERROR_IS_OK(result) && (W_ERROR_V(result) =! W_ERROR_V(WERR_IO_PENDING))) {

Obviously this is meant to be the not-equals operator instead of bitwise not but gcc (version 3 at least) didn't pick it up. Score one for vendor cc.

Wed, 13 Jul 2005

A Blast from the Past

Anna needed to compile some ancient 12-year old Motif software under Linux the other day and ended up trying an install of a really old version of Linux. You would be surprised how much stuff doesn't work under modern versions of GCC, especially C++ where the standard has changed so much. I pulled out my InfoMagic Linux Developer's Resource from March 1995. Here's a list of the contents:

• Complete Linux archives from tsx-11.mit.edu and sunsite.unc.edu
• Complete GNU archive from prep.ai.mit.edu
• Kernel sources up to 1.2.1
• Complete on-line documentation
• DOOM (the game) for LINUX
• Commercial Demos
• Slackware 2.2 with kernel 1.2
• XFree86 Version 3.1.1 (X-Windows)
• Bogus Version 1.0.1
• SLS 1.06 with kernel 1.1.50
• Debian 0.91/3
• JE Distribution and JF (Japanese FAQ's)
• MCC Version 1.0+

Some choice quotes from the Linux Installation and Getting Started guide. On memory requirements:

"Linux requires very little memory to run compared to other advanced operating systems. You should have at the very least 2 megabytes of RAM; however it is strongly recommended that you have 4 megabytes."

"You can run a minimal system with less than 20 megabytes; a complete system with all the bells and whistles in 80 megabytes or less; and a very large system with room for many users and space for future expansion in the range of 100-150 megabytes."

Due to the whole hard disk size issue, Anna had fun tracking down a hard disk old and small enough that would actually work. Also, creating boot floppies(!) from disk images using rawrite was interesting. Those were the days, eh?

Thu, 07 Jul 2005

XMLTV

I've been working on some scripts to manipulate guide data for the PVR. At the moment there's a client to fetch guide data from http://minnie.tuhs.org/tivo-bin/tvguide.pl and a script and database schema to insert the database into a sqlite database. Next step is to write some CGI scripts to browse and manage guide data.

I have this idea that RSS will be an interesting format to express recording information as well as sharing data with other XMLTV users. As with these things, ideas present themselves along the way.

Chris has bought one of the HDTV-Lite DVB cards and has set up MythTV. I think he agrees with me about the excess of user interface crack in the DVB card and channel configuration. (-:

Tue, 21 Jun 2005

Random audio

George Bush Mashups

The infamous GWB mashups. They are much like the Pauline Hanson parodies a few years ago but based on presidential speeches. (Warning: contains political satire and naughty words).

Helen Keller - Electro Ohio LP

Funky experimental electro dance. Very listenable.

Super Mario World Soundtrack

Some guy covers the entire soundtrack of the Super Mario World video game. Now that's just weird.

Sat, 28 May 2005

Kitchen accident

Sun, 15 May 2005

Browsing Working

I have some really simple browsing working now with my Python media server. Selecting any of the video, music, picture and online-media buttons on the Z400 remote brings up a couple of dummy folders. Since my SOAP Browse() implementation always returns the same two folder names, you can go on browsing to an infinite depth, although it appears that the Z400 is limited to 16 levels. I have been working away on some Python classes to represent the Digital Item Declaration Language XML blobs that are returned. It's slowly starting to come together.

I've found a couple more funky bugs:

• The screensaver (bouncing Zensonic logo) isn't enabled when there is an error browsing folders or opening media files.
• The Z400 resets when it receives DIDL declarations that it doesn't like or are inconsistent. Since I'm creating my XML blobs by hand, it's pretty easy to get it wrong. I didn't try and track down exactly what the problem was but it's a sign of only testing your server with your own client.
• If you don't specify a port number for a resource in a browse list the unit attempts to connect to port 0 instead of port 80. Weird but easy to work around.

The next step is to try and get the browse functionality working on a directory of real files and then attempt to get video streaming happening. The code isn't quite ready for a release, although a couple of people have expressed interest in it. I might think about what features would be present for a 0.1 release on sourceforge or something. I would like a little bit of each meda type (video, audio, pictures) to be working for an initial public version.

Fri, 13 May 2005

Python Media Server Resurrected!

I found some time to get my Python Media Server going again. Hopefully there will be some more opportunities over the weekend to get some more hacking done on it. I had to finish off a mini-rewrite of the SSDP server code after a multi-month delay which was a bit tricky since I had reformatted my hard disk in between now and then.

I bought another tuner and popped it in my PC. I think there is a problem with the driver though. I can't seem to get the second card to stream at the same time as the first one which kind of defeats the whole purpose of having two tuners.

Fri, 29 Apr 2005

twisted.internet.gtk2reactor

Here's a short example script that gets Twisted Python, PyGTK and glade together in the same room. I'm getting a bit sick of the single threaded GUI programming. It seems that sooner or later when you're writing a GUI that you will need to address the problem of refreshing when performing a blocking operation, usually accessing the network in some way.

#!/usr/bin/python

from twisted.internet import gtk2reactor
gtk2reactor.install()

import gtk.glade

xml = gtk.glade.XML('foo.glade')

win = xml.get_widget('window')
win.connect('destroy', gtk.main_quit)
win.show()

from twisted.internet import reactor
reactor.run()

Google doesn't know of many people using the gtk2reactor. Hopefully it will work well as I get the impression that the twisted reactor for wxWindows/wxPython doesn't work well and may never do so.

Tue, 26 Apr 2005

Fooling around with valgrind

The new version of valgrind is supposed to have a spiffy new internal representation (IR) format that makes life easier for 64-bit architectures, among other things. It's based on a library, LibVEX, whose purpose in life is to translate from native machine code to IR. The various *grind tools then operate on the generated IR.

Here's a sample of IR courtesy of the --trace-flags option to valgrind:

$ /usr/local/bin/valgrind --tool=memcheck --trace-flags=10000000 /bin/ls

        0x1B8E4C20:  movl %esp,%eax

              ------ IMark(0x1B8E4C20, 2) ------
              PUT(0) = GET:I32(16)

        0x1B8E4C22:  call 0x1B8E4C70

              ------ IMark(0x1B8E4C22, 5) ------
              PUT(56) = 0x1B8E4C22:I32
              t0 = Sub32(GET:I32(16),0x4:I32)
              PUT(16) = t0
              STle(t0) = 0x1B8E4C27:I32
              goto {Call} 0x1B8E4C70:I32

. 0 1B8E4C20 7
. 89 E0 E8 49 00 00 00

The second statement is a bit more complicated. A 32-bit constant address is loaded into the instruction pointer (offset 56 is %eip) and a temporary variable is used to create a stack pointer or maybe a return address that is made when the function call completes. I don't really know that much about x86 assembly language.

Sat, 23 Apr 2005

Finished linux.conf.au

Well it's all over and it was totally rocking. The only major disappointments were that I had a bad case of the flu the entire time (and possibly infected the other delegates) and that I had seen every one of the Best Of talks.

Highlights were Eben Moglen, tridge "reverse engineering" bitkeeper on stage in a few lines of shell, and seeing a bunch of HP people from around Australia and Ft Collins. Eben's talk was almost like a political rally (which I guess in some sense it was) with most of the audience caught in his spell and breaking in to spontaneous applause often.

Muchos thanks to the organisers and volunteers! Congratulations on a very successful conference. Next year's LCA is in Dunedin, New Zealand which looks like a very beautiful location.

Tue, 19 Apr 2005

At linux.conf.au

There's muchos geeking going on and stacks of people sitting around with laptops ignoring each other. Pure bliss!

Sun, 03 Apr 2005

ACPI Startup and Shutdown

One feature I would like to have for my PVR is the ability to shut itself down when there is nothing to do and restart itself when there is something interesting to record. The TiVo is designed to stay on all the time and apparently it does things like dial in to TiVo HQ, reindex its database and other cleanup stuff.

There are quite a few ways I can think of making a server start and restart itself without direct human intervention, but using the ACPI support in the 2.6 kernel looks like the best so far. After some mucking around with BIOS settings, I can suspend my machine with:

# echo 3 > /proc/acpi/sleep

and wake it up at a given time with:

# echo 2005-04-03 20:10:10 > /proc/acpi/alarm

Unbelievably, this works as advertised (2.6.11 kernel) except for the fact that the hardware clock is stored in UTC, not local time. Not bad for an old 800MHz Athlon with a BIOS dated sometime in the year 2000. There were only a few hiccups. The crappy-ass closed source nvidia kernel module refuses to suspend itself, and for some reason mysql can't be shut down by the software suspend code. If I try and unsuspend the machine by hitting the power button, it comes up OK but then decides to immediatly shutdown as the kernel doesn't clear the power button state after resuming. (-:

I recorded some quality British drama by hand last night. It ended up being 2.3GB which is a little under 1MB/sec. This is a widescreen MPEG2 stream as it is broadcast so it's compressed already, but is still quite big. MythTV has various settings for running XviD on received streams in the background. I might implement something like this for archiving recorded shows. I did experience a small amount of filesystem corruption during the recording process. My primary ext3 partition changed to read-only and a reboot+fsck fixed a bunch of stuff. It sounds like there is something dodgy going on with the DVB drivers.

Sun, 27 Mar 2005

Installing MythTV using a DVICO Fusion HDTV DVB-T

The in-kernel DVB drivers work a treat for the DVICO card:

bt878: AUDIO driver version 0.0.0 loaded
bt878: Bt878 AUDIO function found (0).
ACPI: PCI interrupt 0000:00:0d.1[A] -> GSI 9 (level, low) -> IRQ 9
bt878(0): Bt878 (rev 17) at 00:0d.1, irq: 9, latency: 32, memory: 0xd6800000
DVB: registering new adapter (bttv0).
DVB: registering frontend 0 (Zarlink MT352 DVB-T)...

The trick is to do a rmmod snd_bt87x and then a insmod dvb_bt8xx as the standard ordering misdetects the card as bttv0 (analog?).

Jani Talikka has written a funky guide on installing MythTV and configuring it for a DVICO Fusion DVB-T card. The section on installing and fixing the tv_grab_au package was very timely as I was just stuck at that point. There are stacks of missing dependencies - the various XMLTV packages and various perl modules.

I needed to s/askQuestion/ask_choice/g on /usr/local/bin/tv_grab_au to get it to actually work, then follow Jani directions for manually adding channels as scanning for them does not work. More research required to get the XMLTV part of myth working correctly.

I managed to crash the Z400 though. If you send it a HTTP header that does not contain a colon the unit reboots instantly.

Fri, 25 Mar 2005

Installing DVB-T Card

I'm just waiting for a 2.6.11 kernel to compile. This is supposed to have support for the FusionHDTV card. The current kernel that comes with Ubuntu (2.6.8 apparently) doesn't qute know what the card is:

Linux video capture interface: v1.00
bttv: driver version 0.9.15 loaded
bttv: using 8 buffers with 2080k (520 pages) each for capture
bttv: Bt8xx card found (0).
PCI: Found IRQ 9 for device 0000:00:0d.0
PCI: Sharing IRQ 9 with 0000:00:04.2
PCI: Sharing IRQ 9 with 0000:00:04.3
PCI: Sharing IRQ 9 with 0000:00:09.0
PCI: Sharing IRQ 9 with 0000:00:0d.1
bttv0: Bt878 (rev 17) at 0000:00:0d.0, irq: 9, latency: 32, mmio: 0xd7000000
bttv0: subsystem: 18ac:db10 (UNKNOWN)
please mail id, board name and the correct card= insmod option to kraxel@bytesex.org
bttv0: using:  *** UNKNOWN/GENERIC ***  [card=0,autodetected]
bttv0: gpio: en=00000000, out=00000000 in=00ffffff [init]
bttv0: using tuner=-1
bttv0: i2c: checking for MSP34xx @ 0x80... not found
bttv0: i2c: checking for TDA9875 @ 0xb0... not found
bttv0: i2c: checking for TDA7432 @ 0x8a... not found
bttv0: registered device video0
bttv0: registered device vbi0
bt878: AUDIO driver version 0.0.0 loaded
bt878: Bt878 AUDIO function found (0).
PCI: Found IRQ 9 for device 0000:00:0d.1
PCI: Sharing IRQ 9 with 0000:00:04.2
PCI: Sharing IRQ 9 with 0000:00:04.3
PCI: Sharing IRQ 9 with 0000:00:09.0
PCI: Sharing IRQ 9 with 0000:00:0d.0
bt878(0): Bt878 (rev 17) at 00:0d.1, irq: 9, latency: 32, memory: 0xd6800000

While I'm waiting for the kernel modules to compile I'm installing mythtv. It comes nicely packaged into several packages, notably a frontend and backend. Ubuntu is really very nice, although the backspace key in firefox doesn't seem to work at all.

Martin Smith has written a MythTV DVB Setup Guide which I probably should read. I also need to rummage around in the garage and find a spare interior antenna to connect up to the DVB-T card.

On the Canberra PVR scene, two of my friends have taken slightly different approaches. Dave B has bought an Xbox to run Xbox Media Center and Dean has a Mac mini to play shows recorded using an analog tuner card. Who's cusine will reign supreme?

Tue, 15 Mar 2005

DVB-T Card

I bought a DViCO FusionHDTV DVB-T Lite card last week for my PVR project. It allegedly supports SD and HD broadcasts in Australia. There's even a Linux driver written by someone at the University of Queensland. He is supposed to have written the driver using a copy of RegSpy and a multimeter. I am wondering what the multimeter would be used for.

It really does look like the ultimate "budget" card for a PVR box. There's only two two connectors on the rear - antenna in and antenna out/passthrough. Other DVB-T cards have a S-Video or composite input to digitise the output of a VCR or video camera. For a HDTV streaming server, this isn't required.

On the media server front, I have been writing some Python classes to represent the structure of the UPnP ContentDirectory service and routines to generate DIDL-Lite XML to send to the Z400. The exact layout is probably device-specific, but using my cdsclient script the top level folder of the Zensonic Media Server looks like this:

<container id="0\Movie\" parentID="0\" restricted="0" searchable="0" childCount="0">
  <dc:title>Movie</dc:title>
  <upnp:class>object.container</upnp:class>
  <dc:creator></dc:creator>
</container>
<container id="0\Music\" parentID="0\" restricted="0" searchable="0" childCount="0">
  <dc:title>Music</dc:title>
  <upnp:class>object.container</upnp:class>
  <dc:creator></dc:creator>
</container>
<container id="0\Photo\" parentID="0\" restricted="0" searchable="0" childCount="0">
  <dc:title>Photo</dc:title>
  <upnp:class>object.container</upnp:class>
  <dc:creator></dc:creator>
</container>
<container id="0\OnlineMedia\" parentID="0\" restricted="0" searchable="0" childCount="0">
  <dc:title>OnlineMedia</dc:title>
  <upnp:class>object.container</upnp:class>
  <dc:creator></dc:creator>
</container>

Hopefully I can have have a bare-bones server that mimics this layout working fairly soon.

Mon, 07 Mar 2005

More On Test Driven Development

Noel Llopis has some nice introductory blog articles on test driven development as it applies to writing computer games: part1 and part2. Bonus marks for also mentioning scons.

New Z400 Firmware

A new version of the firmware has been released. It also includes a new version of the Zensonic Media Server.

From the release notes:

- Improved Xvid Jitter problem.
- Improve DVB MPEG2 compatibility.
   - File size limit (upgrade from 2G to 4G)
   - Some files that didn't play at all will now play
- Increased maximum image size limit beyond 6 megapixels.
- BUG fixed: CPU utilisation on media server goes to 100% for no reason.
- BUG fixed: Folders with large numbers of files show "Unable to open folder" error

The file size limit is a bit of a worry. 4GB is about four hours of viewing on the Tivo (digitised analog). I would guess that it's a bit less than that for SD and a lot less for HD recordings.

From the forum replies there still seems to be some issues with using the Z400 over wireless.

Sat, 05 Mar 2005

SOAP Client

I've been poking around at the Windows software that comes with the Z400 - the Zensonic Media Server. I was hoping talking to it would be as easy as using the Twisted Python SOAP client but unfortunately that has turned out not to be the case

As far as I can work out, the ZMS has troubles if the Content-Length header is not correct or within a certain range of the HTTP payload length. Also, it craps out if line endings aren't in DOS format (i.e \r\n instead of just \n). I don't know whether this is because the UPnP specification says so, or just by accident. Either way I'll have to figure out how to modify the twisted SOAP client to do what I want.

The next step in my plan is to write some UPnP client tools to make SOAP requests to the ContentDirectory:1 and ConnectionManager:1 services. This should be really useful for working out what the ZMS returns and comparing it with what my media server does.

Tue, 01 Mar 2005

ElementTree

It seems everyone but me has heard of effbot's ElementTree module. It's a lovely little Python module that is "Designed to store hierachical data structures, such as simplified XML infosets, in memory".

Normally I would use the Python XML modules that come with the Python distribution, but ElementTree is just so much nicer. Here's some code to generate the return value for the Z400 browsing the list of internet radio stations:

#!/usr/bin/python

import urllib
from elementtree.ElementTree import Element, SubElement, dump

root = Element('DIDL-Lite')

root.attrib['xmlns'] = 'urn:schemas-upnp-org:metadata-1-0/DIDL-Lite'
root.attrib['xmlns:dc'] = 'http://purl.org/dc/elements/1.1/'
root.attrib['xmlns:upnp'] = 'urn:schemas-upnp-org:metadata-1-0/upnp'

ctr = SubElement(root, 'container')

ctr.attrib['id'] = urllib.quote('0\OnlineMedia\Internet radio\' \
        'Total Country 103 - The Internet\'s NEW Country LEADER ' \
        '(High Bandwidth).pls\\')
ctr.attrib['parentID'] = urllib.quote('0\OnlineMedia\Internet radio\\')
ctr.attrib['restricted'] = '0'
ctr.attrib['searchable'] = '0'
ctr.attrib['childCount'] = '3'

SubElement(ctr, 'dc:title').text = urllib.quote(
        'Total Country 103 - The Internet\'s NEW Country LEADER (High Bandwidth)')

SubElement(ctr, 'upnp:class').text = 'object.container.playlistContainer'
SubElement(ctr, 'dc:creator')
SubElement(ctr, 'res', protocolInfo = 'http-get:*:audio/x-scpls:*',
           size = '480').text = \
        'urllib.quote('http://192.168.1.101:5643/web/' \
        'C:\Program Files\Zensonic Media Server\RadioList\Total Country 103' \
        '- The Internet\'s NEW Country LEADER (High Bandwidth).pls')

dump(root)

I like the way there is no actual mention of anything related to XML. It's all just assigning values to a Python data structure. The program above produces the following disgusting output:

<DIDL-Lite xmlns="urn:schemas-upnp-org:metadata-1-0/DIDL-Lite" xmlns:d
c="http://purl.org/dc/elements/1.1/" xmlns:upnp="urn:schemas-upnp-org:
metadata-1-0/upnp"><container childCount="3" id="0%5COnlineMedia%5CInt
ernet%20radio%5CTotal%20Country%20103%20-%20The%20Internet%27s%20NEW%2
0Country%20LEADER%20%28High%20Bandwidth%29.pls%5C" parentID="0%5COnlin
eMedia%5CInternet%20radio%5C" restricted="0" searchable="0"><dc:title>
Total%20Country%20103%20-%20The%20Internet%27s%20NEW%20Country%20LEADE
R%20%28High%20Bandwidth%29</dc:title><upnp:class>object.container.play
listContainer</upnp:class><dc:creator /><res protocolInfo="http-get:*:
audio/x-scpls:*" size="480">http%3A//192.168.1.101%3A5643/web/C%3A%5CP
rogram%20Files%5CZensonic%20Media%20Server%5CRadioList%5CTotal%20Count
ry%20103%20-%20The%20Internet%27s%20NEW%20Country%20LEADER%20%28High%2
0Bandwidth%29.pls</res></container></DIDL-Lite>

This is then used as as the value of an XML text node after being passed through another level of quoting. (XML encapsulated within XML - WTF were they thinking??). After that the UPnP playback device should have enough information to fetch the .pls file and start streaming audio data.

BTW this XML is documented in the ContentDirectory:1 Service Template Version 1.01, Appendix A.

Mon, 28 Feb 2005

Twisted Python UPnP Example

Anthony Baxter has written a nice little upnp script that runs as a UPnP client (Linux Media Server will run as a UPnP server). It looks for voice over IP gateways by M-SEARCH'ing for urn:schemas-upnp-org:device:InternetGatewayDevice:1. It's a good example of Twisted Python in action.

I've nearly figured out how to get internet radio working on the Z400. Using twisted.web.soap and the SOAPpy modules, my script responds to internet radio browse requests but not with anything sensible yet. I suspect that the contents of the .pls file that describes the radio station is returned in the search response, rather than returning a URL which I think is how audio and video files are served.

Update: fixed broken link as shtoom moved to a different version of viewcvs or something like that.

Sun, 27 Feb 2005

Linux Media Server Detected by Z400

OK I have got some neat stuff working on the Z400 now. My twisted media server listens for and responds to SSDP M-SEARCH datagrams and now appears in the list of servers when you look for media servers on the network. The script points the Z400 at a twisted web server which returns descriptors (big XML files) of the various SOAP endpoints for managing content and connections. The Z400 now bombs out trying to make SOAP remote procedure calls so the next step is to try and implement some of these calls and see what happens.

I did have a lot of trouble trying to get all this working over wireless. Seemingly all the time, the Z400 would not respond to ARP requests and thus TCP connections, SSDP responses, pings etc. I plugged in to the wired port and everything worked immediately. This might explain some problems with the Zensonic Windows media server not being able to see the Z400 that I have noticed when playing around..

There's a topic on the Zensonic Forum for Operating Systems other than Windows, Using the Z400 on Linux/Mac/etc... dated the 17th of September 2004.

"First and foremost, Zensonic Corporation have chosen to release the Z400 to support Microsoft Windows only. This decision remains final"

No replies posted yet. )-: Hopefully the first reply will appear sometime soon!

Also, I need a name for my project. Linux Media Server is kinda boring, and anyway Twisted Python can run on Windows as well as Linux/Unix. I guess I am at Stage 1: Pre-bickering of development in my open source project.

Tue, 22 Feb 2005

Hacking the D-Link DSM-320

According to some random person on the Internet, the z400 is based on the same hardware as the D-Link DSM-320. DSM-320 hacking page can be found here.

More Twisted Python

I still think Twisted Python is the bee's knees. Here's the skeleton of a WBEM server that listens for HTTP POST requests on port 5988 and SSL encrypted ones on port 5989.

#!/usr/bin/python

from twisted.web import server, resource
from twisted.internet import reactor, ssl

class Cimom(resource.Resource):
    isLeaf = 1

    def render_POST(self, request):
        print request.received_headers
        print request.content.readlines()
        return 'xml stuff'

site = server.Site(Cimom())

ssl_context = ssl.DefaultOpenSSLContextFactory(
    '/etc/opt/hp/sslshare/file.pem',
    '/etc/opt/hp/sslshare/cert.pem')

reactor.listenTCP(5988, site)
reactor.listenSSL(5989, site, ssl_context)
              
reactor.run()

Of course it's missing the actual implementation of the server, but now we have all the boring sockets crap out of the way in 20 lines of code. Unfortunately there is a bit of a learning curve in getting started with twisted. Taking a peek in the source to work out what is going on is the best way I have found to get some tricky stuff happening.

UPDATE: Removed logfile parameter to twisted.server.site.Site() as it defaults to no log file.

Sat, 19 Feb 2005

SSDP Snooper

Twisted Python is a great little framework for writing networked clients and servers. Here's a quick little script to listen in on what UPnP announcements the Z400 is making:

#!/usr/bin/python

from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor

class SSDP(DatagramProtocol):
    def datagramReceived(self, data, (host, port)):
        print 'received from %s:%d' % (host, port)
        print data

SSDP_PORT = 1900
SSDP_ADDR = '239.255.255.250'

port = reactor.listenMulticast(SSDP_PORT, SSDP(), SSDP_ADDR)
port.joinGroup(SSDP_ADDR)
reactor.run()

(The tricky bit was performing the joinGroup() as the multicast examples used the "all hosts" address (224.0.0.1) which doesn't require a multicast join.

The Z400 is searching for a UPnP media server:

M-SEARCH * HTTP/1.1
MX: 3
ST: urn:schemas-upnp-org:device:MediaServer:1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"

NOTIFY * HTTP/1.1
LOCATION: http://192.168.1.103:54877/
HOST: 239.255.255.250:1900
SERVER: POSIX, UPnP/1.0, Intel MicroStack/1.0.1497
NTS: ssdp:alive
USN: uuid:PC00:0f:3d:fa:0d:93::upnp:rootdevice
CACHE-CONTROL: max-age=1800
NT: upnp:rootdevice

USN: uuid:AV00:0f:3d:fa:0d:93
USN: uuid:AV00:0f:3d:fa:0d:93::urn:schemas-upnp-org:device:MediaRenderer:1
USN: uuid:AV00:0f:3d:fa:0d:93::urn:schemas-upnp-org:service:AVTransport:1
USN: uuid:AV00:0f:3d:fa:0d:93::urn:schemas-upnp-org:service:ConnectionManager:1
USN: uuid:AV00:0f:3d:fa:0d:93::urn:schemas-upnp-org:service:RenderingControl:1

I guess it's time to look at writing a UPnP media server. Google pops up a couple of proprietary ones: TwonkyVision Media Server, and the Allegro Media Server but no sign of any open source ones.

Fri, 18 Feb 2005

Z400 Installation

I installed my Z400 this morning. I plugged it in to the TV, configured it and updated to the latest firmware over the Internet without too much hassle. As mentioned in one of the reviews, there is no way to configure a ESSID or WEP key using the configuration wizard but this is supposedly fixed in the latest firmware revision.

There's a small amount of user interface crack though. Zensonic's combo-box widget (i.e a list of alternatives of which you can choose one) is a bit hard to use. There is a red ball next to the currently selected option which has really tiny white up and down arrows to indicate that there are more choices to be made. A more traditional implementation, maybe a dropdown list, I think would work better.

The next step is to install the Windows server software in VMWare and start figuring out how to write a Linux replacement. Should be fun. It's supposed to be based on the Universal Plug-and-Play standard which, after a bit of research, looks like some XML over HTTP thingy.

The Programmatic Horseman of the Apocalypse

From Slashdot:

Absolutely every single time I see HN in use, I quickly end up learning that the programmer who wrote the code was an idiot. On at least two separate occasions, I've had to correct C code with HN that did not allocate space for the null terminator on any strings anywhere in the code. That's pretty much the level of programming talent you can expect when you see HN in use.

Conversely, I have never seen any code using HN which I looked at and thought to myself "Well that's a sweet way to solve that problem!" Not even once have I ever seen an elegant algorithm expressed in code written with HN. Not one single time.

As another poster puts it, "I'm Hungarian, and a progammer, you insensitive clod!"

Thu, 17 Feb 2005

PVR Project

Apparently I haven't been updating my blog often enough with no posts for the month of January. (-:

I've decided to build my own PVR. A friend has one of the dual tuner DVB-T Topfield boxes which has amazing quality for free-to-air broadcasts here in Canberra for only about A$800. Last time I priced an equivalent TiVo box (including network card) it came up to about the same price.

The plan at the moment is to build a back-end RAID server running Linux with two or three DVB-T cards that sits in the garage or spare room. Pack it with hard drives and install MythTV and you have a TiVo/Topfield killer.

For viewing recorded programs, the MPEG2/4 files can either be downloaded to a laptop for viewing on planes, hotels etc when travelling, or a set-top box that takes ethernet on one side and TV out on the other. (There's nothing like being able to watch some quality British drama instead of stupid American crap).

I've chosen the Zensonic Z400 as the set top box side of things. It's a great looking unit with wired and wireless ethernet as input, and analong component video, S-Video or composite output. I ordered a unit last week and it has just turned up - very cool. Unfortunately the little sticker that says "warranty void if removed" wasn't stuck on properly and I ripped it trying to stick it down. Grr.