Perl的并行:: ForkManager wait_all_children()采用过长的时间
-
19-09-2019 - |
题
我有一个使用Parallel::ForkManager
的脚本。然而,wait_all_children()过程需要所有的子流程完成后,甚至令人难以置信的很长一段时间。我知道的方法是打印出一些时间戳(见下文)。没有人有任何想法可能是造成这个(我有我的机器上16个CPU核心)?
my $pm = Parallel::ForkManager->new(16);
for my $i (1..16) {
$pm->start($i) and next;
... do something within the child-process ...
print (scalar localtime), " Process $i completed.\n";
$pm->finish();
}
print (scalar localtime), " Waiting for some child process to finish.\n";
$pm->wait_all_children();
print (scalar localtime), " All processes finished.\n";
显然,我会先拿到Waiting for some child process to finish
消息,有,比方说,7:08:35
时间戳。然后,我会得到Process i completed
的邮件列表,与7:10:30
最后一个。不过,我没有收到消息All Processes finished
直到7:16:33
(!)。这是为什么七时10分三十○秒和7时16分33秒之间的6分钟的延迟? THX!
解决方案
我尝试这样:
#!/opt/perl/bin/perl
use strict; use warnings;
use Parallel::ForkManager;
my $pm = Parallel::ForkManager->new(16);
for my $i (1..16) {
$pm->start($i) and next;
sleep rand 20;
printf "%s : Process %d completed\n", scalar localtime, $i;
$pm->finish;
}
printf "%s: Waiting for some child to finish\n", scalar localtime;
$pm->wait_all_children;
printf "%s: All processes finished.\n", scalar localtime;
我得到:
[sinan@archardy Src]$ ./y.pl Thu Mar 11 17:14:16 2010 : Process 3 completed Thu Mar 11 17:14:16 2010: Waiting for some child to finish Thu Mar 11 17:14:18 2010 : Process 8 completed Thu Mar 11 17:14:18 2010 : Process 14 completed <snip>...</snip> Thu Mar 11 17:14:34 2010 : Process 12 completed Thu Mar 11 17:14:34 2010: All processes finished.
我perl 5.10.1
的Linux上并行:: ForkManager 的版本0.7.5
。
因此,我的结论是无论您有正在发生的会发生什么后果问题,当您
# ... do something within the child-process ...
更新:的问题是,要打印的Process finished message
调用之前finish
。尝试下面的版本:
#!/opt/perl/bin/perl
use strict; use warnings;
use Parallel::ForkManager;
my $pm = Parallel::ForkManager->new(16);
$pm->run_on_finish( sub {
printf "%s : Process completed: @_\n", scalar localtime
});
for my $i (1..16) {
$pm->start($i) and next;
sleep rand 20;
$pm->finish;
}
printf "%s: Waiting for some child to finish\n", scalar localtime;
$pm->wait_all_children;
printf "%s: All processes finished.\n", scalar localtime;
有关的更多信息请参见回调并行:: ForkManager文档。如果延迟消失,那么你所观察的症状是由于这样的事实,你是自称派生进程已经完成了它之前完成。
不隶属于 StackOverflow