Log in

The small matter of errors - Luke's Weblog [entries|archive|friends|userinfo]
Luke Gorrie

[ website | My Website ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

The small matter of errors [Jun. 3rd, 2007|10:17 am]
Luke Gorrie
[Tags|, ]

I originally wrote this as an email to Bill Clementson but I decided to post it here too.

Lately there are some Erlang implementations of parallel-map floating around the internet and I just noticed that they don't exhibit satisfactory behaviour in the presence of errors. To see why let's start with an example of the right way for such a function to treat errors:

  lists:map(fun(N) -> 1/N end, [3,2,1,0]).
.. which will neatly raise a division-by-zero exception from the lists:map/2 call. I'd say that it's a bug for a parallel version of map to behave any different.

The version in Joe Armstrong's new book has the bug of using an implicit catch (defensive programming!) to sweep errors under the rug by converting them into return values:

  > pmap(fun(N) -> 1/N end, [3,2,1,0]).

And my own concise implementation:

  pmap(F, L) ->
      Parent = self(),
      [receive {Pid, Result} -> Result end
       || Pid <- [spawn(fun() -> Parent ! {self(), F(X)} end) || X <- L]].
.. is worse because it will hang forever: the child terminates without sending the result and the parent never detects this. (Unfortunately we can't just change spawn to spawn_link because it won't help when the parent is trapping exits.)

I wouldn't say that either is very suitable for use in real programs.

So let's put things right and try to write a proper production-quality implementation of parallel-map that follows lists:map/2 as closely as possible. Here is my new version:

  pmap(F,List) ->
      [wait_result(Worker) || Worker <- [spawn_worker(self(),F,E) || E <- List]].

  spawn_worker(Parent, F, E) ->
      erlang:spawn_monitor(fun() -> Parent ! {self(), F(E)} end).

  wait_result({Pid,Ref}) ->
	  {'DOWN', Ref, _, _, normal} -> receive {Pid,Result} -> Result end;
	  {'DOWN', Ref, _, _, Reason} -> exit(Reason)
Now let's go forth and write parallel programs!

From: (Anonymous)
2008-08-07 09:21 pm (UTC)

Re: capabilities

good questions. i don't have the answer, but i've been lurking on the captalk mailing list trying to slowly learn through osmosis. you might ask there or similar places, i would also be very interested in the answers you might get :-)

(Reply) (Parent) (Thread)