Multi-Threading HttpClient

Question

public class test { 
    public static final int nThreads = 2; 

    public static void main(String[] args) throws ExecutionException, InterruptedException{ 
    // Runnable myrunnable = new myRunnable(); 
     ExecutorService execute = Executors.newFixedThreadPool(nThreads); 

     for (int i = 0; i < nThreads; ++i) { 
      execute.execute(new MyTask()); 
     }   

     execute.awaitTermination(1000, TimeUnit.MILLISECONDS); 

     execute.shutdown(); 
    } 
} 

class MyTask implements Runnable { 
    public static final int maxCalls = 10; 
    public static final int sleepMillis = 500; 
    private static HttpResponse response; 
    private static HttpClient httpclient; 

    public void run(){ 
     int counter = 0; 

     while (true) { 

      if (counter >= maxCalls) { 
       break; 
      } 
      try { 
       Thread.currentThread().sleep(sleepMillis); 
      } catch (InterruptedException e) { 
       // TODO Auto-generated catch block 
       e.printStackTrace(); 
      } 

      execHttpRequest(); 

      ++counter; 
     } 
    } 

    private void execHttpRequest() { 
     httpclient = new DefaultHttpClient(); 
     HttpGet httpget = new HttpGet("My URL"); 

     try { 

      response = httpclient.execute(httpget); 
      BufferedReader br = new BufferedReader(new InputStreamReader(response.getEntity().getContent())); 
      String output; 
      while((output=br.readLine())!=null){ 
       System.out.println(Thread.currentThread().getName() +output); 
      } 
      br.close(); 

      httpclient.getConnectionManager().shutdown(); 
      //httpclient.getConnectionManager().shutdown(); 
     } catch (ClientProtocolException e) { 
      e.printStackTrace(); 
     } catch (IOException e) { 
      e.printStackTrace(); 
     } 
     finally{ 

      httpclient.getConnectionManager().shutdown(); 
     } 

    } 


} 

Durante l'esecuzione di questo codice, ottengo la seguente eccezione:

Exception in thread "pool-1-thread-1" java.lang.IllegalStateException: Invalid use of SingleClientConnManager: connection still allocated. 
Make sure to release the connection before allocating another one. 
    at org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConnManager.java:216) 
    at org.apache.http.impl.conn.SingleClientConnManager$1.getConnection(SingleClientConnManager.java:190) 
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:401) 
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820) 
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754) 
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732) 
    at MyTask.execHttpRequest(test.java:72) 
    at MyTask.run(test.java:60) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
    at java.lang.Thread.run(Thread.java:722) 
java.io.InterruptedIOException: Connection has been shut down 
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:543) 
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820) 
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754) 
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732) 
    at MyTask.execHttpRequest(test.java:72) 
    at MyTask.run(test.java:60) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
    at java.lang.Thread.run(Thread.java:722) 
Caused by: org.apache.http.impl.conn.ConnectionShutdownException 
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.assertValid(AbstractPooledConnAdapter.java:86) 
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.getRoute(AbstractPooledConnAdapter.java:112) 
    at org.apache.http.impl.client.DefaultRequestDirector.establishRoute(DefaultRequestDirector.java:740) 
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:577) 
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425) 
    ... 8 more 

Quando eseguo richiesta http, poi vedo queste eccezioni. Funziona perfettamente bene per single-threaded. Sto cercando di chiamare un URL specifico (che funziona perfettamente) ma quando aggiungo più di un thread, esso genera un'eccezione di stato illegale.

Answer 1

ho capito la mia follia! httpClient e httpRequest erano entrambi statici. Dopo averli resi non statici, funziona perfettamente! Il servizio Executor mi dà un controllo migliore sulla gestione dei thread e sono stato entusiasta di usarlo.

Answer 2

È necessario definire HttpConnectionManager multi-thread, ad es.

MultiThreadedHttpConnectionManager connectionManager = 
            new MultiThreadedHttpConnectionManager(); 
HttpClient client = new HttpClient(connectionManager); 

Per maggiori dettagli è possibile vedere http://hc.apache.org/httpclient-3.x/threading.html

Answer 3

È venuto a dirlo, MultiThreadedHttpConnectionManager è obsoleto. Attualmente (HttpClient versione 4. *) questo è il modo: http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d5e639

SchemeRegistry schemeRegistry = new SchemeRegistry(); 
schemeRegistry.register(
     new Scheme("http", 80, PlainSocketFactory.getSocketFactory())); 

ClientConnectionManager cm = new PoolingClientConnectionManager(schemeRegistry); 
HttpClient httpClient = new DefaultHttpClient(cm); 

Answer 4

Oltre a bpgergo risposta di - Gestione connessioni ha aggiornato ancora una volta (come di HttpClient versione> = 4.3), e ora si dovrebbe usare PoolingHttpClientConnectionManager anziché. Le limitazioni predefinite di PoolingHttpClientConnectionManager sono un totale di 20 connessioni e 2 per route, ma queste possono essere ignorate.

PoolingHttpClientConnectionManager cm=new PoolingHttpClientConnectionManager(); 
cm.setDefaultMaxPerRoute(40); 
cm.setMaxTotal(500); 
CloseableHttpClient client = HttpClients.createMinimal(cm);