process forking in perl

This is a very geeky topic, but I was having a conversation at work today with someone who was trying to do several really simple things in parallel in perl because they didn’t want to wait several minutes for each one to finish. They had knocked up a change to the existing perl, but had tried to do it using perl threads. Normally, that would be fine for most boxes, but when you are running rather old perl on various unixes, including under USS on the IBM z/OS Mainframe, you tend to find out that perl was not compiled with thread support (and recompiling perl for the mainframe is not the easiest thing in the world).

I suggested that they simply use fork() to do the same thing with multiple perl processes, and having not done this in quite a while I tried to search for a good example but couldn’t find anything that looked both complete and simple enough to explain quickly – Hence this post…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#!/usr/bin/perl 
use strict;
use warnings;
 
sub main
{
  my @children;
 
  foreach my $i ( reverse(1..3) )
  {
    my $pid = fork();
 
    if( $pid )
    {
      #If $pid is non zero, then the parent is running
      print "PID $pid forked ($i)\n";
      push(@children, $pid);
    }
    else
    {
      # Else we are a child process ($pid == 0)
      my $rc = child($i);
      exit($rc);
    }
  }
 
  foreach my $n (@children)
  {
    my $pid = waitpid($n,0); # waitpid returns the pid that finished, see perldoc -f waitpid
    my $rc = $? >> 8; # remove signal / dump bits from rc
    print "PID $pid finished with rc $rc\n";
  }
}
 
sub child
{
  my ($arg) = @_;
  print "$arg: start\n";
  sleep $arg;
  print "$arg: end\n";
  return $arg*2;
}
 
main();
exit;

And the output of running the above code is as follows:

PID 16767 forked (3)
PID 16768 forked (2)
PID 16769 forked (1)
3: start
2: start
1: start
1: end
2: end
3: end
PID 16767 finished with rc 6
PID 16768 finished with rc 4
PID 16769 finished with rc 2

It is worth remembering that because each process gets an entire copy of the memory, you have to do more clever things to actually get the processes to talk to each other – but if all you want to do is run something and get a return code back, the above example should get you going pretty quickly.

For what it is worth, the example at https://wiki.bc.net/atl-conf/pages/viewpage.action?pageId=20548191 isn’t too bad actually, and compares perl threads with process forking, and a php example.

About Anton Piatek

I am a Software Developer working for IBM, but a bit of a Ubuntu addict. I love cool tech toys, and am slightly obsessed by photography
This entry was posted in Programming and tagged , . Bookmark the permalink.

3 Responses to process forking in perl

  1. Beanz says:

    Wouldn’t it be better to wait for any child rather than a specific child? The output currently shows the children finishing in the opposite order to the order that they in fact finish. If you do something like:


    ...
    my %children;
    ...
    print "PID $pid forked ($i)\n";
    $children{$pid}++;
    ...
    while (keys %children)
    my $pid = waitpid(-1,0);
    delete $children{$pid};
    ...
    }

    then the output better reflects reality as the children are reaped in the order that they finish.

    I assume you aren’t doing this as the perldoc says your use of waitpid is implemented everywhere but for most people I think the correct ordering it more intuitive. (And I suspect the set of platforms mine doesn’t work on that do have a working fork() call is quite small anyway.)

  2. Anton Piatek says:

    A very good suggestion. In my case I am waiting for all children to finish, but you are right that reaping in the order they finish is better than a pre-defined order.

  3. Beanz says:

    The other advantage is that you don’t leave the earlier finishing children around as zombies while the long running children are still going. Reaping zombies and thus freeing up resources sooner is generally better.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>