Question

I have a code snippet where a loop submits Callable and then checks if these are done and if so print out their Values

    ArrayList<Future<ArrayList<String>>> controllList = new  ArrayList<Future<ArrayList<String>>>();
    System.out.println(""+pagecount);
    for(int n=1;n<=pagecount;n++){
        if(controllList.size()<10){
            Future<ArrayList<String>> temp = exeService.submit(new URLSpider("localhost"));
            controllList.add(temp);
        }
        for(int k=0;k<controllList.size();k++){
            if(controllList.get(k).isDone()){
                System.out.println("Something done");
                ArrayList<String> URLs = controllList.get(k).get();
                for(int h=0;h<URLs.size();h++){
                    System.out.println(URLs.get(h));
                }
                controllList.remove(k);
            }
        }
    }

URLSpider class:

public class URLSpider implements Callable<ArrayList<String>> {
private TagNode node;
private String pageUrl;
private Object[] links;
private ArrayList<String> detailLinks;


public URLSpider(String completePageURL){
    pageUrl = completePageURL;
    detailLinks = new ArrayList<String>();
}
@Override
public ArrayList<String> call() throws Exception {
    HtmlCleaner cleaner = new HtmlCleaner();

    try {
        node = cleaner.clean(new URL(pageUrl));
    } catch (IOException e1) {
        e1.printStackTrace();
    }
    try {
        String link;
        links = node.evaluateXPath("some XPath");
        for(int i =0;i<links.length;i++){
            link=((TagNode)links[i]).getAttributeByName("href");
            System.out.println(link); //the code produces URLs as wanted
            detailLinks.add(link);
        }
    } catch (Exception e) {
        e.printStackTrace();
    }

    return detailLinks;
}

}

My Problem is that the if statement in the second for-Loop never turns true. I checked if the call method reaches the end. And it did so every single time. Also the detailURL list ist full of the wanted URLs. But just the isDone() is false all the time.

May somebody tell me what I am doing wrong here?

Was it helpful?

Solution

Because when you iterate the list, none of the Callables have completed yet.

If you want to block until the Callable is done, you can call controllList.get(k).get() - this will block until the task completes.

For your use case you may also look at the ExecutorCompletionService executor.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top