blocks|key|1597233|text|您可以使用parallelStream()并发调用API。|type|unstyled|depth|inlineStyleRanges|offset|length|style|CODE|entityRanges|data|1597234|List<ResponseEntity<String>>+result+=+employeeList.parallelStream()
++++++++++++.map(id->restTemplate.exchange("http://dummy.restapiexample.com/api/v1/employee/"%2Bid,+HttpMethod.GET,+requestEntity,+String.class))
++++++++++++.collect(Collectors.toList());

result.forEach(entity->saveInDatabase(entity));|code-block|syntax|javascript|1597235|但要注意，parallelStream()可能会耗尽您的应用程序可用的CPU核心。如果应用程序不只执行此任务，而是设计为服务于其他请求，则可能会出现问题。|1597236|因此，按照@davidxxx建议，使用saveAll进行批量插入|1597237|saveAll(result)|1597238|entityMap^0|5|G|0|0|5|G|0|J|7|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@$9|T|A|U|B|C]]|D|@]|E|$]]|$1|F|3|G|5|H|7|V|8|@]|D|@]|E|$I|J]]|$1|K|3|L|5|6|7|W|8|@$9|X|A|Y|B|C]]|D|@]|E|$]]|$1|M|3|N|5|6|7|Z|8|@$9|10|A|11|B|C]]|D|@]|E|$]]|$1|O|3|P|5|H|7|12|8|@]|D|@]|E|$I|J]]|$1|Q|3|-4|5|6|7|13|8|@]|D|@]|E|$]]]|R|$]]

You can use <code>parallelStream()</code> to make API calls concurrently 

<pre><code>List&lt;ResponseEntity&lt;String&gt;&gt; result = employeeList.parallelStream()
 .map(id-&gt;restTemplate.exchange("http://dummy.restapiexample.com/api/v1/employee/"+id, HttpMethod.GET, requestEntity, String.class))
 .collect(Collectors.toList());

result.forEach(entity-&gt;saveInDatabase(entity));
</code></pre>

But beware <code>parallelStream()</code> may also starve the CPU cores available for your application. If the application doesn't make only this task but is designed to serve other requests, it could be an issue. 

So as suggested by @davidxxx use <code>saveAll</code> for batch insert

<pre><code>saveAll(result)
</code></pre>

blocks|key|1597427|text|直接使用Thread+API很容易出错，因为它的级别较低。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1597428|使用parallelStream()可能很有趣，但它也可能是一个问题，因为处理后的流可能会消耗应用程序可用的所有CPU核心。|offset|length|style|CODE|1597429|这意味着您的应用程序的其他HTTP客户端请求可能最近才会得到处理。|1597430|And+note+also+that+the+number+of+threads+used+in+parallelStream()+is+a+JVM+implementation+detail+and+doesn't+make+part+of+the+public+API。|1597431|允许指定池中可用线程数量的ExecutorService+API看起来是一个更好/更健壮的替代方案。|1597432|Spring+Boot提供了一个包装它的built-in+feature。|1597433|您可以将要调用的单个任务提取到方法中，如下所示：|1597434|@Async
public+Future<ResponseEntity<String>>+getEmployee(long+employeeId,++HttpEntity<String>+requestEntity)+{

+++++++String+url+=+"http://dummy.restapiexample.com/api/v1/employee/"+%2B+employeeId;
+++++++ResponseEntity<String>+responseEntity+=+restTemplate.exchange(url,+HttpMethod.GET,+requestEntity,
++++++++++++++++String.class);
++++   return+new+AsyncResult<ResponseEntity<String>>(responseEntity);
+++}|code-block|syntax|javascript|1597435|现在叫它：|1597436|private+void+callApi(List<Long>+employeeList,+HttpEntity<String>+requestEntity)+{

++++++++//+Do+async+calls+and+store+futures+in+a+List
++++++++List<Future<ResponseEntity<String>>>+futures+=+new+ArrayList<>();
++++++++for+(Long+id+:+employeeList)+{
++++++++++++futures.add(getEmployee(id,+requestEntity));++++
++++++++}

++++++++//+Then+process+list+of+future+
++++++++for+(Future<ResponseEntity<String>>+future+:+futures)+
++++++++++++try{+
++++++++++++++++saveInDatabase(future.get());
+++++++++++++++}
+++++++++++++++catch(Exception+e){+
++++++++++++++++++//handle+the+exception
++++++++++++++}

+++++++++}++++
++++}|1597437|顺便说一下，在循环中执行saveInDatabase()操作不是一种正确的方法。|1597438|您希望批处理数据库插入，而不是批处理，因为您有许多数据库插入要做。类似于：|1597439|private+void+callApi(List<Long>+employeeList,+HttpEntity<String>+requestEntity)+{

++List<ResponseEntity<String>>+responseEntities+=+
++++++++employeeList.stream()
++++++++++++++++++++.map(id+->+getEmploye(id))
++++++++++++++++++++.map(future+->+{
++++++++++++++++++++++++++try{return+future.get();}
++++++++++++++++++++++++++++catch(Exception+e){+
++++++++++++++++++++++++++++++++//handle+the+exception
++++++++++++++++++++++++++++++++}
+++++++++++++++++++++++++++++}
+++++++++++++++++++++)
++++++++++++++++++++.collect(toList());+
+++saveInDatabase(responseEntities);++++++++++++
}|1597440|1597441|要使@Asynch功能正常工作，必须在应用程序的@Configuration类上添加@EnableAsync。|1597442|您还可以选择使用适合您需要的池/队列配置来定义Executor+bean。注意:如果您没有定义Spring，Executor将创建一个SimpleAsyncTaskExecutor并使用它(它通过任务创建一个Thread，并且不重用它们)。|1597443|例如：|1597444|@SpringBootApplication
@EnableAsync
public+class+Application+{

++++public+static+void+main(String[]+args)+{
++++++++SpringApplication.run(Application.class,+args);
++++}

++++@Bean
++++public+Executor+taskExecutor()+{
++++++++ThreadPoolTaskExecutor+executor+=+new+ThreadPoolTaskExecutor();
++++++++executor.setCorePoolSize(2);
++++++++executor.setMaxPoolSize(2);
++++++++executor.setQueueCapacity(500);
++++++++executor.setThreadNamePrefix("GithubLookup-");
++++++++executor.initialize();
++++++++return+executor;
++++}


}|1597445|entityMap|0|LINK|mutability|MUTABLE|url|https://stackoverflow.com/questions/30802463/how-many-threads-are-spawned-in-parallelstream-in-java-8|1|https://spring.io/guides/gs/async-method/^0|0|2|G|0|0|1D|G|0|3S|0|0|D|F|0|K|G|1|0|0|0|0|0|C|G|0|0|0|0|2|7|O|E|16|C|0|N|8|1I|8|1V|N|2W|6|0|0|0^^$0|@$1|2|3|4|5|6|7|1P|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|1Q|8|@$D|1R|E|1S|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|1T|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|1U|8|@$D|1V|E|1W|F|G]]|9|@$D|1X|E|1Y|1|1Z]]|A|$]]|$1|L|3|M|5|6|7|20|8|@$D|21|E|22|F|G]]|9|@]|A|$]]|$1|N|3|O|5|6|7|23|8|@]|9|@$D|24|E|25|1|26]]|A|$]]|$1|P|3|Q|5|6|7|27|8|@]|9|@]|A|$]]|$1|R|3|S|5|T|7|28|8|@]|9|@]|A|$U|V]]|$1|W|3|X|5|6|7|29|8|@]|9|@]|A|$]]|$1|Y|3|Z|5|T|7|2A|8|@]|9|@]|A|$U|V]]|$1|10|3|11|5|6|7|2B|8|@$D|2C|E|2D|F|G]]|9|@]|A|$]]|$1|12|3|13|5|6|7|2E|8|@]|9|@]|A|$]]|$1|14|3|15|5|T|7|2F|8|@]|9|@]|A|$U|V]]|$1|16|3|-4|5|6|7|2G|8|@]|9|@]|A|$]]|$1|17|3|18|5|6|7|2H|8|@$D|2I|E|2J|F|G]|$D|2K|E|2L|F|G]|$D|2M|E|2N|F|G]]|9|@]|A|$]]|$1|19|3|1A|5|6|7|2O|8|@$D|2P|E|2Q|F|G]|$D|2R|E|2S|F|G]|$D|2T|E|2U|F|G]|$D|2V|E|2W|F|G]]|9|@]|A|$]]|$1|1B|3|1C|5|6|7|2X|8|@]|9|@]|A|$]]|$1|1D|3|1E|5|T|7|2Y|8|@]|9|@]|A|$U|V]]|$1|1F|3|-4|5|6|7|2Z|8|@]|9|@]|A|$]]]|1G|$1H|$5|1I|1J|1K|A|$1L|1M]]|1N|$5|1I|1J|1K|A|$1L|1O]]]]

Using directly the Thread API is error prone because low level. 
Using <code>parallelStream()</code> may be interesting but it may also be an issue because the processed stream could consume all CPU core available for your application. 
 Which means that other HTTP client requests of your application could be served very lately. 
<a href="https://stackoverflow.com/questions/30802463/how-many-threads-are-spawned-in-parallelstream-in-java-8">And note also that the number of threads used in <code>parallelStream()</code> is a JVM implementation detail and doesn't make part of the public API</a>. 
The <code>ExecutorService</code> API that allows to specify a number of threads available in the pool looks a better/more robust alternative. 

Spring Boot provides a <a href="https://spring.io/guides/gs/async-method/" rel="nofollow noreferrer">built-in feature</a> that wraps it. 
You could extract the individual task to invoke into a method such as :

<pre><code>@Async
public Future&lt;ResponseEntity&lt;String&gt;&gt; getEmployee(long employeeId, HttpEntity&lt;String&gt; requestEntity) {

 String url = "http://dummy.restapiexample.com/api/v1/employee/" + employeeId;
 ResponseEntity&lt;String&gt; responseEntity = restTemplate.exchange(url, HttpMethod.GET, requestEntity,
 String.class);
    return new AsyncResult&lt;ResponseEntity&lt;String&gt;&gt;(responseEntity);
 }
</code></pre>

Now call it : 

<pre><code>private void callApi(List&lt;Long&gt; employeeList, HttpEntity&lt;String&gt; requestEntity) {

 // Do async calls and store futures in a List
 List&lt;Future&lt;ResponseEntity&lt;String&gt;&gt;&gt; futures = new ArrayList&lt;&gt;();
 for (Long id : employeeList) {
 futures.add(getEmployee(id, requestEntity)); 
 }

 // Then process list of future 
 for (Future&lt;ResponseEntity&lt;String&gt;&gt; future : futures) 
 try{ 
 saveInDatabase(future.get());
 }
 catch(Exception e){ 
 //handle the exception
 }

 } 
 }
</code></pre>

As a side note, doing the <code>saveInDatabase()</code> operation into a loop is not a right approach. 
Instead of, you want to batch the database insertions because you have many of them to do. Something like : 

<pre><code>private void callApi(List&lt;Long&gt; employeeList, HttpEntity&lt;String&gt; requestEntity) {

 List&lt;ResponseEntity&lt;String&gt;&gt; responseEntities = 
 employeeList.stream()
 .map(id -&gt; getEmploye(id))
 .map(future -&gt; {
 try{return future.get();}
 catch(Exception e){ 
 //handle the exception
 }
 }
 )
 .collect(toList()); 
 saveInDatabase(responseEntities); 
}
</code></pre>

<hr>

To make the <code>@Asynch</code> feature working, you have to add <code>@EnableAsync</code> on a <code>@Configuration</code> class of your application. 
Optionally you can define a <code>Executor</code> bean with the pool/queue configuration that suit to your need. 
Beware : if you don't define an <code>Executor</code> bean, Spring will create a <code>SimpleAsyncTaskExecutor</code> and use that (it creates a <code>Thread</code> by task and don't reuse them).

For example : 

<pre><code>@SpringBootApplication
@EnableAsync
public class Application {

 public static void main(String[] args) {
 SpringApplication.run(Application.class, args);
 }

 @Bean
 public Executor taskExecutor() {
 ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
 executor.setCorePoolSize(2);
 executor.setMaxPoolSize(2);
 executor.setQueueCapacity(500);
 executor.setThreadNamePrefix("GithubLookup-");
 executor.initialize();
 return executor;
 }


}
</code></pre>

I have around 400k employee ids .I can pass 1 employee id at a time.

How can I call the PI in parallel using Threads so I can improve the performance.
Need some pointers.

saveInDatabase() methods saves the object in a database table.I have marked this method as Synchronized

<pre><code>private void callApi(List&lt;Long&gt; employeeList, HttpEntity&lt;String&gt; requestEntity) {
 Long employeeId;
 for (Long i : employeeList) {

 employeeId = i;// url

 String url = "http://dummy.restapiexample.com/api/v1/employee/" + employeeId;

 ResponseEntity&lt;String&gt; responseEntity = restTemplate.exchange(url, HttpMethod.GET, requestEntity,
 String.class);

 saveInDatabase(responseEntity);

 }
 }
</code></pre>

How to make API calls concurrently

Spring Boot

数据库

我有大约40万个员工id，.I一次可以传递一个员工id。如何使用线程并行调用PI，从而提高性能。需要一些指点。saveInDatabase()方法将对象保存在数据库表中。private void callApi(List<Long> employeeList, HttpEntity<String> requestEnt...

问如何并发进行API调用
EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何并发进行API调用EN

社区

活动

资源

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问如何并发进行API调用
EN