You are viewing lukego

Luke's Weblog - The small matter of errors [entries|archive|friends|userinfo]
Luke Gorrie

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

The small matter of errors [Jun. 3rd, 2007|10:17 am]
Previous Entry Add to Memories Share Next Entry
[Tags|, ]

I originally wrote this as an email to Bill Clementson but I decided to post it here too.

Lately there are some Erlang implementations of parallel-map floating around the internet and I just noticed that they don't exhibit satisfactory behaviour in the presence of errors. To see why let's start with an example of the right way for such a function to treat errors:

  lists:map(fun(N) -> 1/N end, [3,2,1,0]).
.. which will neatly raise a division-by-zero exception from the lists:map/2 call. I'd say that it's a bug for a parallel version of map to behave any different.

The version in Joe Armstrong's new book has the bug of using an implicit catch (defensive programming!) to sweep errors under the rug by converting them into return values:

  > pmap(fun(N) -> 1/N end, [3,2,1,0]).
  [0.333333,
   0.500000,
   1.00000,
   {'EXIT',{badarith,[{erl_eval,eval_op,3},{parmap,do_f,4}]}}]

And my own concise implementation:

  pmap(F, L) ->
      Parent = self(),
      [receive {Pid, Result} -> Result end
       || Pid <- [spawn(fun() -> Parent ! {self(), F(X)} end) || X <- L]].
.. is worse because it will hang forever: the child terminates without sending the result and the parent never detects this. (Unfortunately we can't just change spawn to spawn_link because it won't help when the parent is trapping exits.)

I wouldn't say that either is very suitable for use in real programs.

So let's put things right and try to write a proper production-quality implementation of parallel-map that follows lists:map/2 as closely as possible. Here is my new version:

  pmap(F,List) ->
      [wait_result(Worker) || Worker <- [spawn_worker(self(),F,E) || E <- List]].

  spawn_worker(Parent, F, E) ->
      erlang:spawn_monitor(fun() -> Parent ! {self(), F(E)} end).

  wait_result({Pid,Ref}) ->
      receive
	  {'DOWN', Ref, _, _, normal} -> receive {Pid,Result} -> Result end;
	  {'DOWN', Ref, _, _, Reason} -> exit(Reason)
      end.
Now let's go forth and write parallel programs!
LinkReply

Comments:
[User Picture]From: lukego
2007-06-03 08:48 pm (UTC)

(Link)

My advice is not to worry too much! I learned Erlang from Joe too and I don't reckon you can find a better teacher. Just remember that sometimes the code presented in books and weblogs has been oversimplified a bit to make it more presentable. :-)