My favourite talk of the day was Patrik Nyblom on the development of SMP support in the BEAM virtual machine. He was entertaining andalso inspiring: that is some really hot technology that our programs are sitting on. I'm reminded of Klacke's talk in London last year: Erlang programmers do have a more powerful tool than everybody else and now is the time to make the most of it.

Kostis Sagonas's presentation of the latest Dialyzer developments was great too. I'm one of those opinionated bastards who doesn't care much about standardized programming styles and this tends to put me at odds with people writing linting tools. This time I had no room to complain: the tool is entirely optional, the motivating examples of bugs it can find were very easy to relate to, and it's been developed and maintained as a production tool for many years now. Great marketing!

Some of the other talks really lacked motivating examples for me. I'm sure it's fun to write cloud-hosted databases and map-reduce frameworks, but what problem does it solve for me? The SQL people have long since fallen into the trap of "if all you have is a hammer, everything looks like a nail." NoSQL might be the same.

I dined a bit too well with friends on the nights leading up to the conference and so I missed the first talks about Nitrogen and advanced parse-transform hackery, which is a pity because I heard they were some of the best.

The whole week in Stockholm has been a really great time. I'm writing this on my short flight back to Switzerland at the moment and feeling very glad to live in the neighbourhood again. Now back to Erlang hacking..

My favourite was Dimitri Simon's delightful talk about Piano. He's spent his career writing a program that's great for aeroplane engineers like himself, he developed it using a program that he loves (MCL), and it's maintained with help from the many nice people in the room (Clozure, Edi, Lispworks). He's a one-man company and his licenses start at £50,000. What's not to smile about?

Paul Tarvydas and David McClain each gave forceful talks with a "this is how real engineering is done" flavour, but I wasn't entirely convinced. Do you really need a Lisp-based clone of Erlang with homebrew cryptography to admin routers and farm out calculations? Or a visual digital circuit design environment to mail-merge commercial brochures? They seemed like brilliant and accomplished people with too much opportunity to over-engineer their programs.

Dan Weinreb's talk was an enjoyable taste of Lisp programming in the large. The impression I got was that things are going quite well and their system is coming together nicely. One later reflection though: whereas Lisp was a famous win for their fare search engine, it seemed like more of a mixed bag for their Oracle-Lisp-Java reservation system. I feel for them being unable to separately compile their source files, for example. I look forward to future reports.

I made a four-minute lightning presentation about Openfirmware. I had three things to say: if you want to experiment with low-level programming on a well-engineered self-contained platform, it's easy with Openfirmware on OLPC. If you want to port your Lisp system (or other program) onto a platform that's free of the C/POSIX/Unix legacy, Openfirmware is easy for that too - the only primitive you need is to call Forth's APPLY and then all the drivers are yours. And if you have an operating system of your own (like Movitz) and you want some cheap device drivers, you can write an IEEE 1275 Openfirmware FCODE interpreter and reuse a whole stack of portable ones from Openfirmware. (And if you don't care about any of those things, well, four minutes isn't a very long time.)

The highlight has been a quick cycle tour from near Zürich (Pfäffikon) through Davos and Flüela Pass to Vulpera near the Austrian/Italian borders. Hiked some beautiful mountains, picked lots of wild berries, and got introduced to Kaiserschmarrn at a friendly alp.

I'm extremely happy with my new touring setup: a Specialized road bike, a Deuter Superbike backpack, a waterproof sleeping bag from REI ("just add forest" one-piece camping kit), and bugger all else. France, Germany, Austria, Italy: I'll ride to you soon :-)

Current agenda: ECLM on the weekend and then finding a more permanent home than Juho's couch in Zürich.

code expand-rect  ( src dst w h --- )
   dx  pop              \ Height of source image in pixels
   4 [sp] edi xchg
   8 [sp] esi xchg
   begin
      0 [sp]  cx mov   \ Width of source image in pixels
      begin
         op: ax lods                \ Get a pixel
         op: ax d# 2400 [edi] mov   \ Write to next line
         op: ax stos                \ Write to this line + increment
         op: ax d# 2400 [edi] mov   \ Write to next line
         op: ax stos                \ Write to this line + increment
      loopa
      d# 2400 # edi add             \ Skip the next output line - already written
      edx dec
   0= until
   eax pop   \ Discard source width
   edi pop   \ Restore EDI
   esi pop   \ Restore ESI
c;

Fancy a beer in Zürich? I'll be back after a couple of weeks in east-coast US and Canada. Drop me a line :-)

See you soon. :-)

   h# 25 to node
   h# 290a8 cmd \ high-pass filter, semi-manual mode, 600Hz cutoff
   h# 34001 cmd \ speaker power 1 dB gain
   h# 38001 cmd \ over-current / short-circuit protection, 2.6A threshold
   h# 39019 cmd \ temperature protection at 130C
   h# 42011 cmd \ over-temperature shutdown of class-D

You see, this code is performing pre-setup of a Conexant HDAudio Codec chip to suit the physical peculiarities of the OLPC 1.5's motherboard and other components. The chip itself has no prior knowledge of the machine it's being used in, so some other component has to provide it with useful information like the power of the speakers, their frequency range, the maximum safe level of current, and so on. And so I've adopted these "magic numbers" into my firmware code quite happily -- it's information peculiar to this PC/motherboard and so the firmware is the place to put it.

Here's some more amusing code that's in the firmware: telling the audio chip what it's many and varied pins are connected to on the motherboard, so that it can repeat this information to Linux's audio driver. Once more -- this information can't be baked into the chip, because it's specific to the use of that chip on this particular motherboard.

: port-a  ( -- u )  19 config(  1/8" green left hp-out jack     )config  ;
: port-b  ( -- u )  1a config(  1/8" pink left mic-in jack      )config  ;
: port-c  ( -- u )  1b config(  builtin front mic-in            )config  ;
: port-d  ( -- u )  1c config(  unused line-out                 )config  ;
: port-e  ( -- u )  1d config(  unused line-out                 )config  ;
: port-f  ( -- u )  1e config(  1/8" pink left line-in jack     )config  ;
: port-g  ( -- u )  1f config(  builtin front speaker           )config  ;
: port-h  ( -- u )  20 config(  unused spdiff-out               )config  ;
: port-i  ( -- u )  22 config(  unused spdiff-out               )config  ;
: port-j  ( -- u )  23 config(  unused mic-in                   )config  ;

So I've somewhat improved my mental model of the appropriate ways to factor information between chips, boards, firmwares, and operating systems.

That doesn't mean I'm happy about seeing this kind of information pass over USB. I'd still like to see gadgets designed for simple on-the-wire protocols, as we do on the internet, and take responsibility for as much of their own configuration as possible. I wonder whether I'll change my mind when I start building USB gadgets.. :-)

So Mitch sent me a DIY debricking kit: a replacement SPI FLASH chip with working firmware and some ChipQuick to remove the old chip with. My dad and I heated up a soldering iron and eventually managed to make the transplant. That was fun! But I wouldn't want to do it that way every time :-)

Thankfully my new XO-1.5 boots from something much cooler than an SPI FLASH chip: an Artec FlexyICE ROM emulator attached to the LPC bus. The ROM emulator is seriously cool - it's an FPGA with two interfaces: LPC towards the XO, impersonating a ROM chip, and USB-serial towards my Macbook, receiving new firmware images that I'm creating. It's also open-source hardware and ships with its ~2500 line VHDL sources.

I've wanted to have FPGAs in my life for quite a while now. This feels like a step in the right direction. :-)

Bringups are fun! I would do another :-)

• many: The word many repeats execution of the current line until a key is pressed.

  ok ." luke rules" cr  many
  luke rules
  luke rules
  luke rules
  
  ok see many
  : many   
     key? 0= if    
        0 >in ! 
     then  
  ; 
  

• quine: Since we can introspect the input stream it's easy to write a program that prints itself to stdout.

  ok ( this is a quine ) source type
  ( this is a quine ) source type
  

• patch: The word patch is a simple way to make binary patches to Forth words.

  ok : foo 1 + ;           
  ok see foo
  : foo   
     1 + 
  ; 
  ok 41 foo .
  42 
  
  ok patch - + foo
  ok see foo
  : foo   
     1 - 
  ; 
  ok 43 foo .
  42 
  
  ok patch 5 1 foo
  ok see foo
  : foo   
     5 - 
  ; 
  ok 47 foo .
  42 
  ok 
  

But who can be bothered blogging day-to-day events in the age of twitter?

This turns out to be pretty neat to decode with Erlang's bit syntax:

%% Decode a 3-byte PS/2 mouse report.
%% Return: {Xdelta, Ydelta, LeftButton, MiddleButton, RightButton}
decode_report(<<YO:1,XO:1,YS:1,XS:1,1:1,MMB:1,RMB:1,LMB:1,X:8,Y:8>>) ->
    {<<DX:10/signed>>,<<DY:10/signed>>} = {<<XS:1,XO:1,X:8>>, <<YS:1,YO:1,Y:8>>},
    {DX, DY, LMB, MMB, RMB}.

Here's a test function:

test_decode_report() ->
    {42+256, 42-512, 1, 0, 1} = decode_report(<<2#01101011, 42, 42>>).

~$ hg
Traceback (most recent call last):
  File "/usr/local/bin/hg", line 18, in 
    mercurial.util.set_binary(fp)
  File "/Library/Python/2.5/site-packages/mercurial/demandimport.py",
  line 74, in __getattribute__
    self._load()
  File "/Library/Python/2.5/site-packages/mercurial/demandimport.py",
  line 46, in _load
    mod = _origimport(head, globals, locals)
  File "/Library/Python/2.5/site-packages/mercurial/util.py",
  line 33, in 
    _encoding = locale.getlocale()[1]
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/locale.py",
  line 460, in getlocale
    return _parse_localename(localename)
  File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/locale.py",
  line 373, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

;;; lively.el version 0.1 --- interactively updating text
;;; Copyright 2009 Luke Gorrie <luke@bup.co.nz>
;;;
;;; Go to the end of any of the following lines and run `M-x lively'
;;;   Current time:      (current-time-string)
;;;   Last command:      last-command
;;;   Open buffers:      (length (buffer-list))
;;;   Unix processes:    (lively-shell-command "ps -a | wc -l")
;;;
;;; then the code will be replaced by its formatted result -- and
;;; periodically updated. You can create little dashboards.
;;; Use `M-x lively-stop' to restore order.
;;;
;;; Based on the Squeak hack by Scott Wallace.

(require 'cl)

(defvar lively-overlays nil "List of all overlays representing lively text.")
(defvar lively-timer    nil "Idle timer for updating lively text.")
(defvar lively-interval 0.25 "Idle time before lively text update in seconds.")

(defun lively ()
  "Make the expression before point lively."
  (interactive)
  (lively-region (save-excursion (backward-sexp) (point)) (point)))

(defun lively-region (start end)
  "Make the region lively."
  (interactive "r")
  (when (null lively-timer)
    (lively-init-timer))
  (push (make-overlay start end) lively-overlays))

(defun lively-update ()
  "Update the display of all visible lively text."
  (interactive)
  (dolist (o lively-overlays)
    (when (get-buffer-window (overlay-buffer o))
      (condition-case err
          (lively-update-overlay o)
        (error (message "Error in lively expression: %S" err)
               (lively-delete-overlay o))))))

(defun lively-delete-overlay (o)
  (delete-overlay o)
  (setq lively-overlays (remove o lively-overlays)))

(defun lively-update-overlay (o)
  "Evaluate the lively code for O and update its display text."
  (with-current-buffer (overlay-buffer o)
    (let ((expr (buffer-substring (overlay-start o) (overlay-end o))))
      (overlay-put o 'display (format "%s" (eval (read expr)))))))

(defun lively-init-timer ()
  "Setup background timer to update lively text."
  (setq lively-timer (run-with-timer 0 lively-interval 'lively-update)))

(defun lively-stop ()
  "Remove all lively regions in Emacs."
  (interactive)
  (when lively-timer (cancel-timer lively-timer))
  (setq lively-timer nil)
  (mapc 'delete-overlay lively-overlays)
  (setq lively-overlays nil))

;;; Nice to have:

(defun lively-shell-command (command)
  "Execute COMMAND and return the output, sans trailing newline."
  (let ((result (shell-command-to-string command)))
    (substring result 0 (1- (length result)))))

• Keyboard and mouse: they support a simplified bootstrap mode that's very simple to deal with. Here's a driver for the common parts of bootstrap-mode keyboard and mouse:

  USBHID : USBDevice ()
  
  USBHID init: devicename
  [
    super init: (OFW open: devicename).
    self setConfiguration: 1.
    self setBootstrapProtocol.
  ]
  
  USBHID setBootstrapProtocol
  [
    self controlSetIndex: 0 value: 0
         requestType: (DR_HIDD bitOr: DR_OUT) request: SET_PROTOCOL
  ]
  
  USBHID setIdle: ms
  [
    self controlSetIndex: 0 value: (ms // 4 << 8)
         requestType: (DR_HIDD bitOr: DR_OUT) request: SET_IDLE
  ]
  
  USBHID selftest
  [
    self init.
    1000 timesRepeat: [ self readReport ifNotNilDo: [ :e | e println]. ]
  ]
  

  and here's a subclass that knows how to turn the reports into mouse events:

  USBMouse : USBHID ()
  
  USBMouse init         [ self init: '/mouse' ]
  USBMouse init: device [ super init: device. self setIdle: 0 ]
  
  USBMouse readReport   [ ^MouseEvent fromUSB: (self intrIn: 1) ]
  
  MouseEvent fromUSB: report
  [
    self := self new.
    left  := (report first bitAnd: 1) ~~ 0.
    right := (report first bitAnd: 2) ~~ 0.
    dx := report second.
    dy := report third.
    "Treat bit 8 as sign."
    (dx bitAnd: 0x80) ~~ 0 ifTrue: [ dx := dx - 256 ].
    (dy bitAnd: 0x80) ~~ 0 ifTrue: [ dy := dy - 256 ].
  ]
  

  The keyboard is similar: it reports the set of keys (identified by scancode) that are currently pressed, and uses a fixed mapping from scancodes onto key identifiers.

  I regret that I haven't come back to look at the full-blown USB Human Input Device protocol yet!

• Ethernet: In theory the USB Communications Device Class specifies the high-level interface for ethernet-like network adapters, but in practice most USB-ethernet dongles seem to be built from an ASIX USB-ethernet chip. For example, the Apple USB-ethernet dongle is internally an ASIX 88772A.

  ASIX chips aren't compatible with the USB CDC standard, they use their own custom protocol instead. The protocol is simple but it's lower-level than CDC: you're sometimes peeking and poking registers instead of exchanging messages.

• Display: My previous posts (first, second) describe the USB-VGA adapter I found. I'm unhappy with this device: taking a notoriously closed graphics chip and blindly bridging its PCI bus onto USB is really kludgy. The components don't even seem to be particularly cheap (Octopart suggests $15 for the USB-PCI bridge chip) so I suppose the intention was to save development money by not having to program the dongle. I'd be happier if they'd spent the same money on an ARM- or FPGA- based dongle and programmed it to do the job gracefully, but that's me being a naive software guy.

ret |= SETIREGAND(SISCR, 0x5b, 0xdf);
ret |= SETIREG(SISSR, 0x05, 0x86);
ret |= SETIREGOR(SISSR, 0x20, 0x01);

ret |= SETREG(SISMISCW, 0x67);

for (i = 0x06; i <= 0x1f; i++) {
	ret |= SETIREG(SISSR, i, 0x00);
}
for (i = 0x21; i <= 0x27; i++) {
	ret |= SETIREG(SISSR, i, 0x00);
}

There's no sense in filling up source files with boring numbers, so I changed strategy. I recorded what Linux sends over USB to make the dongle boot and checked whether I could just blindly replay exactly the same traffic. This turned out to be easy and fun!

1. Capture USB traffic while plugging in the dongle:

   tcpdump -w trace.pcap -ni usb0
   

2. Open the trace in wireshark and Print as ascii like this:

   Frame 3 (14 bytes on wire, 14 bytes captured)
   USB URB
       URB id: 0x00000000de478380
       URB type: URB_SUBMIT ('S')
       URB transfer type: URB_BULK (3)
       Endpoint: 0x0e
       Device: 3
       URB bus id: 1
       Device setup request: not present ('-')
       Data: present (0)
       URB status: Operation now in progress (-EINPROGRESS) (-115)
       URB length [bytes]: 6
       Data length [bytes]: 6
       [Response in: 4]
       [bInterfaceClass: Unknown (0xffff)]
       Application Data: 420044D00000
   

3. Filter and massage the printed trace:

   #!/usr/bin/env awk -f
   /Endpoint:/                     { ep = "" }
   /Endpoint: 0x0[de]/             { ep = strtonum($2) }
   /Application Data:/ && ep != "" { printf("SND %.2x %s\n", ep, $3); }
   

   to resemble a script:

   SND 0d 1F002403000004000000
   SND 0d 1F006403000004000000
   SND 0d 1F008403000004000000
   SND 0d 1F000001000000070000
   SND 0d 0F0004000000
   ...
   

4. Write a traffic-replaying interpreter for the trace in (for example) Forth:

   d# 128 constant /buffer
   /buffer dma-alloc value buffer   \ buffer in DMA-friendly memory.
   
   0 value line
   : SND ( "endpoint" "data" -- )
      line 1+ to line
      get-hex#                      ( endpoint )
      safe-parse-word decode-hex    ( endpoint adr len )
      dup >r                        ( r: len )
      buffer swap                   ( endpoint adr buffer len )
      cmove                         ( endpoint )
      buffer r> rot                 ( adr len endpoint )
      bulk-out ?dup if
         ." usb error on write = " .  ." line " line . abort
      then
   ;
   

5. Then plug the dongle into my OLPC XO and execute the Forth code in Openfirmware and -- drumroll -- the dongle boots! I see the contents of the framebuffer appear on my monitor. Cool!

Beats writing it by hand :-)

There are three big chips on the circuit board:

• EM638325TS-6G: This is RAM -- the framebuffer will be here. (The framebuffer has to be in the dongle -- USB isn't fast enough to feed SVGA.)
• Graphics: There's a chip hidden behind a metal shield, but on the PCB we can see it connected to the RAM and the VGA port, so it's the video chip. The Linux sources suggest it's a SiS315(E) and I don't see any reason to doubt.
• NET2280 REV1-LF: PCI-USB bridge. AHA!

I like this world below the OS -- firmware, chips, microcontrollers, circuit boards, FPGAs, Verilog, etc. I'm very happy to have the chance to explore it :-)

Forth tips would be welcome. :-)

I have xoos talking to my USB mouse and keyboard with the bootstrap-protocol and that was easy enough. I'm working in a non-public repository right now so I'll post the code later when more is working.

$ echo -ne '         U GC_init\n0000000a t TTFont___5fdebugName\n' > f
$ cat f
         U GC_init
0000000a t TTFont___5fdebugName
$ cat f | grep -w T               # This gets it WRONG!
0000000a t TTFont___5fdebugName
$ cat f | grep -w T | grep -w T   # but this is right!
$ grep --version
GNU grep 2.5.3