Blogs

SpringSource Blog

Spring MVC 3.2 Preview: Introducing Servlet 3, Async Support

Rossen Stoyanchev

Last updated on November 5th, 2012 (Spring MVC 3.2 RC1)

Overview

Spring MVC 3.2 introduces Servlet 3 based asynchronous request processing. This is the first of several blog posts covering this new capability and providing context in which to understand how and why you would use it.

The main purpose of early releases is to seek feedback. We've received plenty of it both here and in JIRA since this was first posted after the 3.2 M1 release. Thanks to everyone who gave it a try and commented! There have been numerous changes and there is still time for more feedback!

At a Glance

From a programming model perspective the new capabilities appear deceptively simple. A controller method can now return a java.util.concurrent.Callable to complete processing asynchronously. Spring MVC will then invoke the Callable in a separate thread with the help of a TaskExecutor. Here is a code snippet before:

// Before
@RequestMapping(method=RequestMethod.POST)
public String processUpload(final MultipartFile file) {
    // ...
    return "someView";
}

// After
@RequestMapping(method=RequestMethod.POST)
public Callable<String> processUpload(final MultipartFile file) {

  return new Callable<String>() {
    public Object call() throws Exception {
      // ...
      return "someView";
    }
  };
}

A controller method can also return a DeferredResult (new type in Spring MVC 3.2) to complete processing in a thread not known to Spring MVC. For example reacting to a JMS or an AMQP message, a Redis notification, and so on. Here is another code snippet:

@RequestMapping("/quotes")
@ResponseBody
public DeferredResult<String> quotes() {
  DeferredResult<String> deferredResult = new DeferredResult<String>();
  // Add deferredResult to a Queue or a Map...
  return deferredResult;
}

// In some other thread...
deferredResult.setResult(data);
// Remove deferredResult from the Queue or Map

The above samples lead to many questions and we'll get to more details in subsequent posts. For now, I'll begin by providing some context around these features.

Motivation for Asynchronicity In Web Applications

The most basic motivation for asynchronicity in web applications is to handle requests that take longer to complete. Maybe a slow database query, a call to an external REST APIs, or some other I/O-bound operations. Such longer requests can exhaust the Servlet container thread pool quickly and affect scalability.

In some cases you can return to the client immediately while a background job completes processing. For example sending an email, kicking off a database job, and others represent fire-and-forget scenarios that can be handled with Spring's @Async support or by posting an event to a Spring Integration channel and then returning a confirmation id the client can use to query for the results.

In other cases, where the result is required, we need to decouple processing from the Servlet container thread or else we'll exhaust its thread pool. Servlet 3 provides just such support where a Servlet (or a Spring MVC controller) can indicate the response should be left open after the Servlet container thread is exited.

To achieve this, a Servlet 3 web application can call request.startAsync() and use the returned AsyncContext to continue to write to the response from some other separate thread. At the same time from a client's perspective the request still looks like any other HTTP request-response interaction. It just takes longer to complete. The following is the sequence of events:

  1. Client sends a request
  2. Servlet container allocates a thread and invokes a servlet in it
  3. The servlet calls request.startAsync(), saves the AsyncContext, and returns
  4. The container thread is exited all the way but the response remains open
  5. Some other thread uses the saved AsyncContext to complete the response
  6. Client receives the response

There is of course a lot more to the Servlet async support. You can find various examples and writeups, but the above sums up the basic, minimum concept you need to know. The next post covers a second motivation for asynchronous request processing — the need for browsers to receive information updates in real time.

Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • Twitter
 

66 responses


  1. Great post Rossen!

    This is the feature I was waiting for long time. Some alternatives already exist (Atmosphere, Comet …) but their integration with Spring needs some work.

    One question though: is the AsyncContext in the sequence of events saved in the Http session ? If yes how's about scalability (big fat session)? If no, where do you save it ?


  2. @Duyhai, glad to hear it. As shown above there are two ways to initiate async processing — by returning a Callable or a DeferredResult — both store nothing in the HTTP session. The comment in the second example mentions storing a DeferredResult in a Map or Queue (for use from another thread) but I'll defer the details on that to the subsequent posts coming.


  3. Can't wait for your next blog post…I hope it will contain the word: websocket.


  4. Zart, it does contain the word websocket but no such support yet, see and track SPR-9356.


  5. Hi Rossen,

    I have a question regarding the motivation. You are talking about supporting long running requests, but what is the essential difference with processing the request async in a Spring component (service) and processing it in a Web controller. So:

    1. Spring 3.1: request -> controller -> component (@async)

    vs

    2. Spring 3.2: request -> controller (async) -> component

    In scenario 1, is the request also not being blocked right? But the async method is executed at a different level.

    Cheers, Stephan


  6. @Stephan
    Both scenario 1 and 2 will be scalable because your controller methods can complete quickly and the hard work is done in a separate thread.

    However, in scenario 2 you also get the benefit of leaving the response open after your controller method completes. This allows you to keep writing to it from the other thread. That is not possible in scenario 1.


  7. @Flemming

    thanks for clarifying!

    Cheers, Stephan


  8. Will there be something like ThreadLocal for these requests, but that spans the whole request? Like RequestLocal or something similar.


  9. Piotr, the short answer is yes and more detail will be provided on various aspects of the lifecycle of a Spring MVC request.


  10. This is fantastic. There are so many areas where this will be extremely useful.


  11. I don't mind whether the Websockets is there or not.

    All I want is a way to notify the web layer when a new event occurs on server-side. Currently it can be achieved with frameworks like Atmosphere or Comet but their integration with Spring is not that easy.

    An built-in solution for this need is welcomed.


  12. Hi Rossen,

    Greate Post!

    I tried your example for asychronous support. The container didn't start the new thread and didn't invoke the call() method. However it did print the request async support=true and async started=ture. After that its irgnoring return Callable() statement and expecting "process.jsp" as a view. So I am getting 404 error.

    I configured async-supported=true for DispatchServlet in the web.xml.

    Here is my controller method :

    @RequestMapping(method = RequestMethod.POST, value="process")
    public Callable process(final Model model, HttpServletRequest request) {
    System.out.println("request asyncronouus support=" request.isAsyncSupported());
    System.out.println("request asyncronouus started=" request.isAsyncStarted());
    return new Callable() {
    public String call() throws Exception {
    System.out.println("entered call() method");
    // Process input file..
    //Thread.sleep(5000L);
    String processName = _service.process("process");
    model.addAttribute("message", "processMVC():name=" processName);
    return "index";
    }
    };

    }

    Please let me know what is the issue here.
    when you guys are expecting to release these changes?


  13. Ravusree, assuming you're on version 3.2.0.BUILD-SNAPSHOT, make sure that you're also using the new @MVC support classes.

    The changes are already available in the latest snapshot and you can run them in the spring-mvc-async branch of the spring-mvc-showcase. The M1 release should be out in a couple of weeks.


  14. I rather excited about this feature however I think it will require more work outside of the MVC layer to get it to work. At one point I was even trying to make my own AnnotationHandler that used Atmosphere till I ran into the ThreadLocal issue.

    As mentioned earlier the ThreadLocal lifecycle is what concerns me most. I'm really hoping that this feature perhaps surfaces a generalized solution for Threadlocal management.

    Right now Threadlocal is used in the TransactionSynchronizationManager, Spring Security and for some JSF things. Each of theses manage threadlocal in their own way. I believe we need a sort of Subscriber/Publisher pattern where different things know when to load/unload their Threadlocal data.

    Whatever the solution I believe for complete usability with the existing spring framework some solution to the threadlocal issue needs to be addressed.


  15. Adam, as part of a typical request, attributes may be bound to the current (container) thread at various points such as the FrameworkServlet, filters, and interceptors. In an async scenario we still need to unbind attributes from the container thread but also apply the same bind and unbind actions to the thread used for async processing.

    We recognized this as something fundamental and have made updates to all places in Spring MVC where ThreadLocal attributes may be created have been taken care of (e.g. FrameworkServlet, OpenSessionInViewFilter/Interceptor, etc.) As a result those should just work as expected.

    In brief the mechanism involves registering a java.util.concurrent.Callable that can bind and unbind ThreadLocal attributes to a thread if necessary. Obviously we haven't modified any external code but the mechanism can be used outside of Spring, for example in Spring Security or even in application code.

    This is exactly the kind of feedback we're looking for so do give it a try in the context of your application(s) and let us know!


  16. Hi Rossen

    I have a question. Can i cancel this AsyncRequest from another Request.Think of this usecase.I have a high-computing action and i want to provide cancel option for this.How can i achieve this using Async support?


  17. Sathis, the Callable you return can be any implementation, including one that contains cancellation logic. Simply hold on to the reference you return and invoke the cancellation logic when you need to.


  18. How does this feature compare to Asynchronous Controller in ASP.NET?


  19. Sorry, forgot to include http://msdn.microsoft.com/en-us/library/system.web.mvc.asynccontroller.aspx to reference question regarding comparing to the Asp.Net MVC's AsyncController class and AsyncManager class.


  20. Joe, I'm not familiar with the AsyncController of ASP.NET. It looks like it provides base class methods around the lifecycle of an async request. In a Spring MVC @Controller there is no base class. Instead we interpret the method arguments and the return values to decide what the controller needs. So for example you can choose to return a Callable/DeferredResult or not, hence dynamically making a decision whether to process asynchronously or not.

    Aside from that, the feature described here builds on the model provided by the Servlet API so I imagine that's where some more fundamental differences may arise. If you have more familiarity with ASP.NET and would like to write a blog post, that would be very useful.


  21. Rossen,

    How would I be able to do something like the following: http://forum.springsource.org/showthread.php?126633-Handling-a-Multipart-File-Upload-in-a-separate-thread


  22. Hi Rossen,
    This is exactly what i was looking for. I have started looking at PlayFramework. How is this new @Async support compared to what PlayFramework provides?

    My Questions are
    1) Are you using JBoss's NettyIO?
    2) Who should manage the thread pool that is used to execute async requests?
    3) Is this how the whole model looks like?
    a) client sends request
    b) Servlet thread starts processing.
    c) Servlet thread makes an Async remote Rest call.
    d) Servlet thread returns to serve other http connections.
    e) Remote Http Call returns and an async thread is created to pack construct the response and sends it to the client.
    Thanks
    sri


  23. I am not able to enable async request support. Even when I add true in my web.xml in element I still have an exception: "org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalStateException: Async support must be enabled on a servlet and for all filters…"
    Of course I use Spring 3.2.0.BUILD-SNAPSHOT.


  24. Assuming this is in a Servlet 3 container, perhaps you're not using the RequestMappingHandlerAdapter? See this note here.


  25. My base is Spring template MVC project. To be able to use DeferredResult I had to change Spring version to 3.2.0.BUILD-SNAPSHOT. I'm not sure (I'm a beginer), but I suppose that I don't use any of the mentioned support classes directly (these introduced in Spring 3.1 and earlier). The exception appears when I return DeferredResult as if there was no async-supported set to true in my web.xml, but it is there in servlet element.


  26. I just tried a template project and it worked for me. All I had to do is change to 3.2.0.BUILD-SNAPSHOT, modify web.xml to declare 3.0 in the top element, and modify the servlet declaration to have the async-supported sub-element with a value of "true". Keep in mind that if you have any filters those have to have async-supported as well.


  27. Today I tried once again (at work, Windows machine) on clean project and still without positive result. Then (at home) I made a try on Linux and it was fine. What is also strange for me is that on my Linux machine I had to add repository with spring snapshots to my pom to resolve 3.2.0.BUILD-SNAPSHOT while on Windows there was no such need. It is strange because I have the same STS version on both PCs and I assumed that template MVC project is exactly the same on both. Now I have to find how to set the custom timeout…


  28. Configuring the timeout is currently easier with the MVC Java config than with the MVC namespace. There is a plan to make it easier to configure with both, see SPR-9399. Otherwise you can check the Spring reference documentation, the section on advanced customizations with the MVC namespace.


  29. I still can not make it work on my Windows PC. Here is my test project:
    http://www.speedyshare.com/gXPkE/AsyncTest.zip . There are only two new controller functions. First is mapped to: /quote/{customerId} and returns DeferredResult. Second one is mapped to: /handleQuote/{customerId} and allows to set the result.


  30. Rather than continuing the discussion here, it would be more appropriate if you started a forum thread. I would only suggest confirming whether a basic Servlet 3 async sample, without using Spring MVC, succeeds or not. That will isolate the possibility of any environment issues.


  31. I have imported spring-mvc-async branch and I have the same problem for example when I press @ResponseBody in Writing Responses tab -> http://postimage.org/image/j6dnsip1n/ . It has to be some environment issue, but it is strange because it's clean STS.


  32. Hi Rossen,

    I've tried running the spring-mvc-async from a STS IDE VmWare vFabric tc Server Developer Edition and I am getting this exception.

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalStateException:
    Async support must be enabled on a servlet and for all filters involved in async request processing.
    This is done in Java code using the Servlet API or by adding
    "true" to servlet and filter declarations in web.xml.
    at org.springframework.web.servlet.FrameworkServlet.finalizeProcessing(FrameworkServlet.java:977)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:923)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:801)

    Please help. Thank you

    Regards,
    Alvin


  33. I believe there's a bug in DeferredResult#handle. The IllegalStateException should be thrown when this.initializationLatch.await() returns false (which happens on a time out), and not when it throws InterruptedException (which happens someone calling interrupt() on the thread).


  34. Thanks for pointing that out.


  35. Hello Rossen,

    I tried to run the SpringSource-spring-mvc-showcase-8b9e275 example without altering anything and I get this exception:

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalStateException: Async support must be enabled on a servlet and for all filters involved in async request processing. This is done in Java code using the Servlet API or by adding "true" to servlet and filter declarations in web.xml.
    org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:927)
    org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:815)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:641)
    org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:789)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:722)

    I run the example on VMware vFabric tc Server Developer Edition v2.7.

    Can you please advise?

    Regards,

    Julien.


  36. That is strange. I presume you're running the code from the spring-mvc-async branch? I just tried it myself on Tomcat 7.0.30 and it works fine. The async-supported flag is enabled in web.xml.

    https://github.com/SpringSource/spring-mvc-showcase/blob/spring-mvc-async/src/main/webapp/WEB-INF/web.xml#L26


  37. Hello Rossen,
    I tried the same commit with Tomcat 7.0.30 and it does work.
    Thanks,


  38. Rossen,

    Further to my prior email, I must mention that I don't see any such line as this

    16:19:23 [SimpleAsyncTaskExecutor-1] DispatcherServlet

    in the Tomcat 7.0.30 console.

    Also, the browser hangs for the 3 seconds specified in the call() method.

    Is this the expected behavior?

    Julien.


  39. The logging may have changed since M1. By default the asynchronous thread should output with the tag that looks like [MvcAsync1].

    Yes, the browser is expected to hang in this example. The request certainly lasts (and remains open) for 3 seconds. The next post goes into a lot more detail on that.


  40. There certainly wasn't any [MvcAsyncX] log in my Tomcat console. Only the usual synchronous tomcat logs. Never mind. I'll wait for the next article/post about this very interesting topic. Regards.


  41. Hey there! Pretty neat stuff. I'm playing with M2 and running into a problem. I'm use DeferredResult in my controller, with AJAX as the source of the HTTP request. However, I'm noticing that normal jQuery doesn't know how to handle responses that use Transfer-Encoding: chunked.

    While deferring the result, I still want normal headers to be sent back, not chunked. Any way to override the Transfer-Encoding to make it acceptable to the browser (without having to use sockets, etc.)? or another approach I should be taking?


  42. Please ignore my previous post. It has nothing to do with chunked encoding. What I was experiencing was making multiple requests to the same REST endpoint in the same browser. It appears all browsers make requests to the same url in a sequential manner, due to caching implementations. Nothing wrong with Spring. :-)


  43. How does spring webflow fit in this async model? So. lets say I want to defer the whole execution from a action.
    Or will this feature be only for MVC use cases?


  44. The feature is supported explicitly in Spring MVC @Controller classes but the underlying async support can be incorporated in Spring Web Flow as well in a future release.


  45. Thx for the quick reply. I think it would be quite useful if that feature is added in the webflow. I can smell Callable maybe.
    On a separate topic, you briefly mention about binding and unbinding thread locals both in container and spring/async thread and looking from the source[OpenSessionInViewInterceptor] looks like one can use a combination of CallableProcessingInterceptor and the AsyncWebRequestInterceptor to achieve the binding/unbinding. I may be missing something but looking at WebAsyncManager though it calls postprocess from finally block to the CallableInterceptorChain, the chain itself returns/short circuits if one the interceptors throws an exception in postProcess. Does this mean postprocess in some interceptors may get skipped and would not result in resource cleaning? it is possible I read it wrong. but wanted to clarify. Thx


  46. Good catch! I just committed a fix in the CallableInterceptorChain.


  47. Great. Thx!
    And I meant to say in my previous comment "Callable Event" for spring webflows..


  48. Hi,

    I thought the idea of comet was to allow a large number of requests to remain open, where the cost of each request was just the socket (i.e. not thread per connection, which is much more expensive). While this feature makes the requests async from the perspective of tomcat threads, there is still a thread per request (presumably spring is managing a cachedThreadPool)? Is the point of this feature really just to hack an unbounded thread pool on top of tomcat's fixed thread pool?


  49. Indeed with a Callable, the remaining processing is done in a dedicated thread essentially replacing a thread in Tomcat's thread pool for a thread in a more specialized thread pool. By specialized I mean configured for a more specific purpose. With a DeferredResult however, there is no automatic thread created and processing can be finalized on any thread, e.g. a JMS event listener can update any number of DeferredResult instances. The spring-mvc-chat sample and the AMQP stocks sample in later posts work that way.


  50. @Rossen I'm personally (you asked the community for feedback) much more interested in DeferredResult aka AMQP/JMS support. Especially AMQP as I think Spring's RabbitMQ support was a good move and we use/love RabbitMQ. It is superior to JMS.

    The alternative of just returning some Future and putting into a different Thread pool does not interest me. An out of the box RabbitMQ with Spring MVC chat like app I think would be a killer demo. Its also whats missing as Atmosphere already has "Broadcasters" for Redis and JMS.

    I can help participate in writing an example or app (or even maven archetype) if others are not already working on it.


  51. Adam, you've probably seen the modified Spring AMQP stocks sample? If you'd like to put together a different sample or possibly extend the Spring MVC chat sample that would be most welcome!


  52. Can DeferredResult object be reused?


  53. I want to use a ringbuffer to permanently reuse the DefferedResult object, but I find that that object once calling setResult(), and can't call that a second time.
    Can it provide a clear() method to reuse itself?


  54. What do you have in mind with reusing it? The DeferredResult is created while processing a specific request and it can be used to provide a result to complete the processing at a later time. Once the response is completed, the DeferredResult can no longer be used.


  55. Hi Rossen:

    We can not get this sample to work. can you please post a minimalism code wich the client does an ajax call from the browser which makes a long pooling request of 10 minutes and the server serves a string in 2 minute interval.
    If you can please include all the libraries the project needs to run, so we could just import it.

    israel


  56. There are examples in the subsequent posts, post #3, post #4, and post #5.


  57. Hi Rossen, Please have a look the thread from stackoverflow.

    http://stackoverflow.com/questions/15381837/spring-asyncronous-processing-does-not-return-to-view

    It does not display any async processing.


  58. Greets,

    How can i jusy return an HttpStatus on success from an async hbase put?
    By using DeferredResult ?

    I have an async serverside proceasing -a db write.
    On fail i will just call deferredRes.onError(throwable) and that will ttansparently go thru my apring configures ExceptionResolver to figure out some 50x return status.

    Now i just wonder how to efficiently return a 204 on success….or how to redirect to other controller method annotated to return 204..for tjat i use SmartView namely RrdirectView?


  59. To ret 204 on sucess i just use DefereedResult ?


  60. What type T other than String is supported by DeferredResult generic type?
    I would use a HttpStatus for sucess..or a RedirectView to a controller merhod which would return 204 status


  61. The DeferredResult generic type can be any type allowed in an @RequestMapping method (see javadoc for the annotation). It replaces the return value type of that method in order to enable providing the value later (from a different thread). So RedirectView, yes but HttpStatus is not a valid return type. ResponseEntity is what you can use to control the response status.


  62. Hi. The following responds 200 on second dispatch spring says "Null modelView, assuming done".
    When debugging, i saw that the deferredResult had no resultHandler set.

    @RequestMapping(value = STREAM_PATH, method = RequestMethod.POST)
    @ResponseBody
    public DeferredResult<ResponseEntity> heartbeat(@PathVariable(DEVICE_ID) String deviceId,..){
    ..
    final DeferredResult<ResponseEntity> deferredResult = new DeferredResult<ResponseEntity>();
    heartRateMonitor.heartbeat(deviceId, streamId, duration, until).setFutureMutationListener(
    new FutureMutationListener() {
    @Override
    public void done() {
    deferredResult.setResult(new ResponseEntity("", HttpStatus.NO_CONTENT));
    }

    @Override
    public void failedOn(Throwable e) {
    deferredResult.setErrorResult(e);
    }
    });
    }
    Please help:)


  63. Hi Rossen,
    Actually I noticed that it returns 200 from the start, much before the deferredRes callback is actually called back from the backend thread :(


  64. "Null modelView, assuming done" in combination with ResponseEntity makes sense. The ResponseEntity return value is handled in a way similar to @ResponseBody and in both cases there is no view resolution and the request is effectively handled before the HandlerAdapter returns to the DispatcherServlet.

    What is the problem actually?


  65. It executes sincrnously and returns 200 instantly without callback being called yet. When i reach ok from async backend put i set 204 code but of course the resp was already commited.


  66. Okay that sounds like a possible configuration issue. You'll need to provide more detail but this isn't the right place. You can compare to one of the several examples provided in this post series (spring-mvc-chat, spring-mvc-showcase, etc) or create a small self-contained example and make it available some place.

5 trackbacks

Leave a Reply