"And more code (no need to wait)" by petrovg

Fri, 23 Nov 2007 01:53:18 PST

Thanks to Yuri for pointing out that you don’t need to wait until all the results are done to start processing them. Here, we’ve only got integers, but in reality it is likely that the individual results will be big and meaningful and actually take some time to interpret or do something with. To start working on them as soon as each one is ready, we use CompletionService:


public class ParallelCompletionCounter {
    public int countWordsAsSoonAsFileFinishes(List files) {
        CompletionService completionService = new ExecutorCompletionService(Executors.newFixedThreadPool(4));
        for(final File file : files) {
            Callable counter = new Callable() {
                public Integer call() {
                    System.out.println("Starting file " + file.getName() + " at " + System.currentTimeMillis());
                    int time = 500;
                    if (file.getName().contains("304")) {
                        time = 5000;
                    }
                    try {
                        Thread.sleep(time);
                    } catch (InterruptedException e) {
                        // TODO Auto-generated catch block
                        e.printStackTrace();
                    }
                    WordCounter counter = new WordCounter();
                    int count =  counter.count(file);
                    System.out.println("Done file " + file.getName() + " at " + System.currentTimeMillis());
                    return count;
                }
            };
            completionService.submit(counter);
        }
        try {
            int totalCount = 0;
            for (int i = 0; i<4; i++) {
                totalCount += completionService.take().get();
                System.out.println("Added File Count");
            }
            return totalCount;
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        } catch (ExecutionException e) {
            throw new RuntimeException(e);
        }
    }

    public static void main(String[] args) {
        long start = System.currentTimeMillis();
        ParallelCompletionCounter counter = new ParallelCompletionCounter();
        List files = new ArrayList(args.length);
        for (String filename : args) {
            files.add(new File(filename));
        }
        System.out.println("Total word count is " + counter.countWordsAsSoonAsFileFinishes(files));
        System.out.println("Total elapsed time " + (System.currentTimeMillis() - start));
    }   
}

Supple software comments on And more code (no need to wait)

"And more code (no need to wait)" by petrovg