Luke Gorrie (lukego) wrote,
Luke Gorrie
lukego

The small matter of errors

I originally wrote this as an email to Bill Clementson but I decided to post it here too.

Lately there are some Erlang implementations of parallel-map floating around the internet and I just noticed that they don't exhibit satisfactory behaviour in the presence of errors. To see why let's start with an example of the right way for such a function to treat errors:

  lists:map(fun(N) -> 1/N end, [3,2,1,0]).
.. which will neatly raise a division-by-zero exception from the lists:map/2 call. I'd say that it's a bug for a parallel version of map to behave any different.

The version in Joe Armstrong's new book has the bug of using an implicit catch (defensive programming!) to sweep errors under the rug by converting them into return values:

  > pmap(fun(N) -> 1/N end, [3,2,1,0]).
  [0.333333,
   0.500000,
   1.00000,
   {'EXIT',{badarith,[{erl_eval,eval_op,3},{parmap,do_f,4}]}}]

And my own concise implementation:

  pmap(F, L) ->
      Parent = self(),
      [receive {Pid, Result} -> Result end
       || Pid <- [spawn(fun() -> Parent ! {self(), F(X)} end) || X <- L]].
.. is worse because it will hang forever: the child terminates without sending the result and the parent never detects this. (Unfortunately we can't just change spawn to spawn_link because it won't help when the parent is trapping exits.)

I wouldn't say that either is very suitable for use in real programs.

So let's put things right and try to write a proper production-quality implementation of parallel-map that follows lists:map/2 as closely as possible. Here is my new version:

  pmap(F,List) ->
      [wait_result(Worker) || Worker <- [spawn_worker(self(),F,E) || E <- List]].

  spawn_worker(Parent, F, E) ->
      erlang:spawn_monitor(fun() -> Parent ! {self(), F(E)} end).

  wait_result({Pid,Ref}) ->
      receive
	  {'DOWN', Ref, _, _, normal} -> receive {Pid,Result} -> Result end;
	  {'DOWN', Ref, _, _, Reason} -> exit(Reason)
      end.
Now let's go forth and write parallel programs!
Tags: erlang, hacking
Subscribe

  • blockfreq, C++, Lua and random thoughts

    I'm still hopping programming langauges. Cool that after so much hacking there are still new ones that are actually important to learn. Juho…

  • EUC 2009

    The Erlang User Conference in Stockholm was a lot of fun! Thank you everybody at Erlang Training and Consulting for the excellent organisation.…

  • Travels

    I'm headed to Stockholm next week, and Boston the week after. I'd be happy to have the traditional tuesday Man in the Moon pub night in Stockholm if…

  • Post a new comment

    Error

    default userpic
    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 10 comments

  • blockfreq, C++, Lua and random thoughts

    I'm still hopping programming langauges. Cool that after so much hacking there are still new ones that are actually important to learn. Juho…

  • EUC 2009

    The Erlang User Conference in Stockholm was a lot of fun! Thank you everybody at Erlang Training and Consulting for the excellent organisation.…

  • Travels

    I'm headed to Stockholm next week, and Boston the week after. I'd be happy to have the traditional tuesday Man in the Moon pub night in Stockholm if…