I'm headed to Stockholm next week, and Boston the week after. I'd be happy to have the traditional tuesday Man in the Moon pub night in Stockholm if anybody's in town, and Xach's already put the word out that Lispers should come for dinner at the Cambridge Brewing Company on friday august 7th (6pm).

See you soon. :-)

Magic numbers

Today I wrote some device driver code of precisely the sort I'm always complaining about -- containing seemingly opaque magic numbers that have no relevance to the operating system and "should have been" built into the hardware:
   h# 25 to node
   h# 290a8 cmd \ high-pass filter, semi-manual mode, 600Hz cutoff
   h# 34001 cmd \ speaker power 1 dB gain
   h# 38001 cmd \ over-current / short-circuit protection, 2.6A threshold
   h# 39019 cmd \ temperature protection at 130C
   h# 42011 cmd \ over-temperature shutdown of class-D
and, for once, I actually see why this is the (firmware) device driver's responsibility.

You see, this code is performing pre-setup of a Conexant HDAudio Codec chip to suit the physical peculiarities of the OLPC 1.5's motherboard and other components. The chip itself has no prior knowledge of the machine it's being used in, so some other component has to provide it with useful information like the power of the speakers, their frequency range, the maximum safe level of current, and so on. And so I've adopted these "magic numbers" into my firmware code quite happily -- it's information peculiar to this PC/motherboard and so the firmware is the place to put it.

Here's some more amusing code that's in the firmware: telling the audio chip what it's many and varied pins are connected to on the motherboard, so that it can repeat this information to Linux's audio driver. Once more -- this information can't be baked into the chip, because it's specific to the use of that chip on this particular motherboard.

: port-a  ( -- u )  19 config(  1/8" green left hp-out jack     )config  ;
: port-b  ( -- u )  1a config(  1/8" pink left mic-in jack      )config  ;
: port-c  ( -- u )  1b config(  builtin front mic-in            )config  ;
: port-d  ( -- u )  1c config(  unused line-out                 )config  ;
: port-e  ( -- u )  1d config(  unused line-out                 )config  ;
: port-f  ( -- u )  1e config(  1/8" pink left line-in jack     )config  ;
: port-g  ( -- u )  1f config(  builtin front speaker           )config  ;
: port-h  ( -- u )  20 config(  unused spdiff-out               )config  ;
: port-i  ( -- u )  22 config(  unused spdiff-out               )config  ;
: port-j  ( -- u )  23 config(  unused mic-in                   )config  ;
I wonder if anyone will make use of this knowledge of what colour the audio jacks are!

So I've somewhat improved my mental model of the appropriate ways to factor information between chips, boards, firmwares, and operating systems.

That doesn't mean I'm happy about seeing this kind of information pass over USB. I'd still like to see gadgets designed for simple on-the-wire protocols, as we do on the internet, and take responsibility for as much of their own configuration as possible. I wonder whether I'll change my mind when I start building USB gadgets.. :-)


I bricked my OLPC XO the other week by flashing bad stuff onto the SPI FLASH. That's the 1MB flash chip that the XO boots from -- it's supposed to contain the Embedded Controller software image and Openfirmware, but when either of those is messed up then the machine won't function and can't fix itself.

So Mitch sent me a DIY debricking kit: a replacement SPI FLASH chip with working firmware and some ChipQuick to remove the old chip with. My dad and I heated up a soldering iron and eventually managed to make the transplant. That was fun! But I wouldn't want to do it that way every time :-)

Thankfully my new XO-1.5 boots from something much cooler than an SPI FLASH chip: an Artec FlexyICE ROM emulator attached to the LPC bus. The ROM emulator is seriously cool - it's an FPGA with two interfaces: LPC towards the XO, impersonating a ROM chip, and USB-serial towards my Macbook, receiving new firmware images that I'm creating. It's also open-source hardware and ships with its ~2500 line VHDL sources.

I've wanted to have FPGAs in my life for quite a while now. This feels like a step in the right direction. :-)


So I was lucky enough to join in the OLPC XO 1.5 bringup at Quanta in Taipei. This was great fun: I wrote an Openfirmware HDAudio driver and we used it to test and debug parts of the main board. I got to watch people doing cool things with soldering irons, oscilloscopes, LPC ROM emulators, and other implements of destruction. Mitch debugged the memory controller interactively using a Forth that fits strictly in cache - that's a nice trick.

Bringups are fun! I would do another :-)

Forth school

I went to Forth school with Mitch Bradley yesterday. Here are a few cool new things I learned:
  • many: The word many repeats execution of the current line until a key is pressed.
    ok ." luke rules" cr  many
    luke rules
    luke rules
    luke rules
    ok see many
    : many   
       key? 0= if    
          0 >in ! 
  • quine: Since we can introspect the input stream it's easy to write a program that prints itself to stdout.
    ok ( this is a quine ) source type
    ( this is a quine ) source type
  • patch: The word patch is a simple way to make binary patches to Forth words.
    ok : foo 1 + ;           
    ok see foo
    : foo   
       1 + 
    ok 41 foo .
    ok patch - + foo
    ok see foo
    : foo   
       1 - 
    ok 43 foo .
    ok patch 5 1 foo
    ok see foo
    : foo   
       5 - 
    ok 47 foo .
I love Forth.


Exciting things lately - visited people along the east coast of Australia, went to JAOO in Brisbane, bricked my OLPC XO by installing a bad firmware, my dad and I unbricked it by soldering on a replacement SPI FLASH chip containing good firmware (thanks Mitch!), now getting to put my Forth where my mouth is and fly over to Taipei to get in the way of help with OLPC 1.5 hardware bringup.

But who can be bothered blogging day-to-day events in the age of twitter?

PS/2 Mouse Decoding

Me and Ian Piumarta have been playing around with PS/2 mouse drivers and partly in Erlang. We came up with a neat way to parse the 3-byte report from a mouse that describes its position-change and button states. Here's the PS/2 mouse report format:

Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1 Bit 0
Byte1 Yoverflow Xoverflow Ysign Xsign 1 MiddleBtn RightBtn LeftBtn
Byte2 X movement
Byte3 Y movement

This turns out to be pretty neat to decode with Erlang's bit syntax:

%% Decode a 3-byte PS/2 mouse report.
%% Return: {Xdelta, Ydelta, LeftButton, MiddleButton, RightButton}
decode_report(<<YO:1,XO:1,YS:1,XS:1,1:1,MMB:1,RMB:1,LMB:1,X:8,Y:8>>) ->
    {<<DX:10/signed>>,<<DY:10/signed>>} = {<<XS:1,XO:1,X:8>>, <<YS:1,YO:1,Y:8>>},
    {DX, DY, LMB, MMB, RMB}.
The DX and DY values are really 10-bit signed numbers, with the top two bits placed in the header byte, and the bit syntax has no trouble converting 10 bits from funky sources into signed numbers.

Here's a test function:

test_decode_report() ->
    {42+256, 42-512, 1, 0, 1} = decode_report(<<2#01101011, 42, 42>>).

My first experience with Mercurial

~$ hg
Traceback (most recent call last):
  File "/usr/local/bin/hg", line 18, in 
  File "/Library/Python/2.5/site-packages/mercurial/",
  line 74, in __getattribute__
  File "/Library/Python/2.5/site-packages/mercurial/",
  line 46, in _load
    mod = _origimport(head, globals, locals)
  File "/Library/Python/2.5/site-packages/mercurial/",
  line 33, in 
    _encoding = locale.getlocale()[1]
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/",
  line 460, in getlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/",
  line 373, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

lively.el 0.1

Long time no Emacs hack!
;;; lively.el version 0.1 --- interactively updating text
;;; Copyright 2009 Luke Gorrie <>
;;; Go to the end of any of the following lines and run `M-x lively'
;;;   Current time:      (current-time-string)
;;;   Last command:      last-command
;;;   Open buffers:      (length (buffer-list))
;;;   Unix processes:    (lively-shell-command "ps -a | wc -l")
;;; then the code will be replaced by its formatted result -- and
;;; periodically updated. You can create little dashboards.
;;; Use `M-x lively-stop' to restore order.
;;; Based on the Squeak hack by Scott Wallace.

(require 'cl)

(defvar lively-overlays nil "List of all overlays representing lively text.")
(defvar lively-timer    nil "Idle timer for updating lively text.")
(defvar lively-interval 0.25 "Idle time before lively text update in seconds.")

(defun lively ()
  "Make the expression before point lively."
  (lively-region (save-excursion (backward-sexp) (point)) (point)))

(defun lively-region (start end)
  "Make the region lively."
  (interactive "r")
  (when (null lively-timer)
  (push (make-overlay start end) lively-overlays))

(defun lively-update ()
  "Update the display of all visible lively text."
  (dolist (o lively-overlays)
    (when (get-buffer-window (overlay-buffer o))
      (condition-case err
          (lively-update-overlay o)
        (error (message "Error in lively expression: %S" err)
               (lively-delete-overlay o))))))

(defun lively-delete-overlay (o)
  (delete-overlay o)
  (setq lively-overlays (remove o lively-overlays)))

(defun lively-update-overlay (o)
  "Evaluate the lively code for O and update its display text."
  (with-current-buffer (overlay-buffer o)
    (let ((expr (buffer-substring (overlay-start o) (overlay-end o))))
      (overlay-put o 'display (format "%s" (eval (read expr)))))))

(defun lively-init-timer ()
  "Setup background timer to update lively text."
  (setq lively-timer (run-with-timer 0 lively-interval 'lively-update)))

(defun lively-stop ()
  "Remove all lively regions in Emacs."
  (when lively-timer (cancel-timer lively-timer))
  (setq lively-timer nil)
  (mapc 'delete-overlay lively-overlays)
  (setq lively-overlays nil))

;;; Nice to have:

(defun lively-shell-command (command)
  "Execute COMMAND and return the output, sans trailing newline."
  (let ((result (shell-command-to-string command)))
    (substring result 0 (1- (length result)))))


Here's what I've discovered about some kinds of USB device:
  • Keyboard and mouse: they support a simplified bootstrap mode that's very simple to deal with. Here's a driver for the common parts of bootstrap-mode keyboard and mouse:
    USBHID : USBDevice ()
    USBHID init: devicename
      super init: (OFW open: devicename).
      self setConfiguration: 1.
      self setBootstrapProtocol.
    USBHID setBootstrapProtocol
      self controlSetIndex: 0 value: 0
           requestType: (DR_HIDD bitOr: DR_OUT) request: SET_PROTOCOL
    USBHID setIdle: ms
      self controlSetIndex: 0 value: (ms // 4 << 8)
           requestType: (DR_HIDD bitOr: DR_OUT) request: SET_IDLE
    USBHID selftest
      self init.
      1000 timesRepeat: [ self readReport ifNotNilDo: [ :e | e println]. ]
    and here's a subclass that knows how to turn the reports into mouse events:
    USBMouse : USBHID ()
    USBMouse init         [ self init: '/mouse' ]
    USBMouse init: device [ super init: device. self setIdle: 0 ]
    USBMouse readReport   [ ^MouseEvent fromUSB: (self intrIn: 1) ]
    MouseEvent fromUSB: report
      self := self new.
      left  := (report first bitAnd: 1) ~~ 0.
      right := (report first bitAnd: 2) ~~ 0.
      dx := report second.
      dy := report third.
      "Treat bit 8 as sign."
      (dx bitAnd: 0x80) ~~ 0 ifTrue: [ dx := dx - 256 ].
      (dy bitAnd: 0x80) ~~ 0 ifTrue: [ dy := dy - 256 ].
    The keyboard is similar: it reports the set of keys (identified by scancode) that are currently pressed, and uses a fixed mapping from scancodes onto key identifiers.

    I regret that I haven't come back to look at the full-blown USB Human Input Device protocol yet!

  • Ethernet: In theory the USB Communications Device Class specifies the high-level interface for ethernet-like network adapters, but in practice most USB-ethernet dongles seem to be built from an ASIX USB-ethernet chip. For example, the Apple USB-ethernet dongle is internally an ASIX 88772A.

    ASIX chips aren't compatible with the USB CDC standard, they use their own custom protocol instead. The protocol is simple but it's lower-level than CDC: you're sometimes peeking and poking registers instead of exchanging messages.

  • Display: My previous posts (first, second) describe the USB-VGA adapter I found. I'm unhappy with this device: taking a notoriously closed graphics chip and blindly bridging its PCI bus onto USB is really kludgy. The components don't even seem to be particularly cheap (Octopart suggests $15 for the USB-PCI bridge chip) so I suppose the intention was to save development money by not having to program the dongle. I'd be happier if they'd spent the same money on an ARM- or FPGA- based dongle and programmed it to do the job gracefully, but that's me being a naive software guy.
I'm finding driver writing frustratingly slow going. I'm used to a very experimental and exploratory programming style, but that's not very effective because when you make a mistake the devices won't tell you where! This reminds me of talking to SIM cards and web services. I'm looking for a smarter working style.