Blogs

SpringSource Blog

Spring MVC 3.2 Preview: Making a Controller Method Asynchronous

Rossen Stoyanchev

Last updated on November 5th, 2012 (Spring MVC 3.2 RC1)

In previous posts I introduced the Servlet 3 based async capability in Spring MVC 3.2 and discussed techniques for real-time updates. In this post I'll go into more technical details and discuss how asynchronous processing fits into the Spring MVC request lifecycle.

As a quick reminder, you can make any existing controller method asynchronous by changing it to return a Callable. For example a controller method that returns a view name, can return Callable<String> instead. An @ResponseBody that returns an object called Person can return Callable<Person> instead. And the same is true for any other controller return value type.

A central idea is that all of what you already know about how a controller method works remains unchanged as much as possible except that the remaining processing will occur in another thread. When it comes to asynchronous execution it's important to keep things simple. As you'll see even with this seemingly simple programming model change, there is quite a bit to consider.

The spring-mvc-showcase has been updated for Spring MVC 3.2. Have a look at CallableController. Method annotations like @ResponseBody and @ResponseStatus apply to the return value from the Callable as well, as you might expect. Exceptions raised from a Callable are handled as if they were raised by the controller, in this case with an @ExceptionHandler method. And so on.

If you execute one of the CallableController methods through the "Async Requests" tab in the browser, you should see output similar to the one below:

08:25:15 [http-bio-8080-exec-10] DispatcherServlet - DispatcherServlet with name 'appServlet' processing GET request for [...]
08:25:15 [http-bio-8080-exec-10] RequestMappingHandlerMapping - Looking up handler method for path /async/callable/view
08:25:15 [http-bio-8080-exec-10] RequestMappingHandlerMapping - Returning handler method [...]
08:25:15 [http-bio-8080-exec-10] WebAsyncManager - Concurrent handling starting for GET [...]
08:25:15 [http-bio-8080-exec-10] DispatcherServlet - Leaving response open for concurrent processing
08:25:17 [MvcAsync1] WebAsyncManager - Concurrent result value [views/html]
08:25:17 [MvcAsync1] WebAsyncManager - Dispatching request to resume processing
08:25:17 [http-bio-8080-exec-6] DispatcherServlet - DispatcherServlet with name 'appServlet' resumed processing GET request for [...]
08:25:17 [http-bio-8080-exec-6] RequestMappingHandlerMapping - Looking up handler method for path /async/callable/view
08:25:17 [http-bio-8080-exec-6] RequestMappingHandlerMapping - Returning handler method [...]
08:25:17 [http-bio-8080-exec-6] RequestMappingHandlerAdapter - Found concurrent result value [views/html]
08:25:17 [http-bio-8080-exec-6] DispatcherServlet - Rendering view [...] in DispatcherServlet with name 'appServlet'
08:25:17 [http-bio-8080-exec-6] JstlView - Added model object 'fruit' of type 1
08:25:17 [http-bio-8080-exec-6] JstlView - Added model object 'foo' of type 1
08:25:17 [http-bio-8080-exec-6] JstlView - Forwarding to resource [/WEB-INF/views/views/html.jsp]
08:25:17 [http-bio-8080-exec-6] DispatcherServlet - Successfully completed request

Notice how the initial Servlet container thread exits quickly after logging a message that concurrent handling has started. That's because the controller method returned a Callable. A second thread — managed by Spring MVC through an AsyncTaskExecutor — invokes the Callable to produce a value, in this case a String-based view name, and then the request is dispatched back to the Servlet container. Finally, in a third Servlet container thread (the dispatch), processing is completed by rendering the selected view. If you look at the timestamps you'll notice a 2 second, simulated delay between when the initial thread exits and the Callable is ready.

Note: if you are not familiar with the Servlet 3 async API, an async dispatch is similar to forwarding except a forward occurs in the same thread while a dispatch is used from an application thread to resume processing in a servlet container thread.

TaskExecutor Configuration

By default Spring MVC uses a SimpleAsyncTaskExecutor to execute Callable instances returned by controller methods. For production you must replace it with an AsyncTaskExecutor implementation configured appropriately for your environment. The MVC Java config and the MVC namespace both provide options to configure an AsyncTaskExecutor and async request processing in general. You can also configure the RequestMappingHandlerAdapter directly.

Timeout Value

If an async request does not complete processing within a certain amount of time, the Servlet container raises a timeout event and if not handled, the response is completed. You can configure the timeout value through the MVC Java config and the MVC namespace, or directly on the RequestMappingHandlerAdapter. If not configured, the timeout value will depend on the underlying Servlet container. On Tomcat it is 10 seconds and it starts after the initial Servlet container thread exits all the way out.

MvcAsyncTask

What if you want to customize the timeout value or the task executor for a specific controller method? For such occasions, you can wrap the Callable in an instance of MvcAsyncTask. The constructor of MvcAsyncTask accepts a timeout value and a task executor. Furthermore, it provides onTimeout and onCompletion methods that allow you to register for "timeout" and "completion" callbacks. Like "finally" in a try-catch block, "completion" always takes place when an async request completes. The "timeout" callback occurs prior to "completion" and can select an alternative value to use to complete processing as well as notify the Callable to stop processing.

The following is the sequence of events in a timeout scenario:

  1. Controller method returns a Callable wrapped in an MvcAsyncTask
  2. Spring MVC begins execution of the Callable in a separate thread
  3. The Servlet container thread exits (and the timeout period begins)
  4. MvcAsyncTask is notified of a callback
  5. The callback code selects an alternative value and notifies the Callable to cancel processing
  6. The request is dispatched back to the container to complete processing with the alternate value

To fully understand the above scenario consider the threads involved — the initial Servlet container thread where request processing begins, the Spring MVC managed thread where the Callable executes, the Servlet container thread in which the timeout event is raised, and the Servlet container thread processing the final async dispatch.

Exceptions

When an Exception is raised by a Callable, it is handled through the HandlerExceptionResolver mechanism just like exceptions raised by any other controller method. The more detailed explanation is that the exception is caught and saved, and the request is dispatched to the Servlet container where processing resumes and the HandlerExceptionResolver chain invoked. This also means that @ExceptionHandler methods will be invoked as usual.

Handler Interception

The preHandle method of a HandlerInterceptor is invoked as usual from the initial Servlet container thread. If the controller returns a Callable and async processing starts, there is neither a result nor is the request complete. Therefore postHandle and afterCompletion are not invoked in the initial Servlet container thread. Instead interceptors can implement AsyncHandlerInterceptor, a sub-interface, and the afterConcurrentHandlingStarted method. After the Callable is done and the request dispatched to the Servlet container, all methods of the HandlerInterceptor are invoked in the dispatched thread.

Servlet Filters

All Spring Framework Servlet filter implementations have been modified as necessary to work in asynchronous request processing. As for any other filters, some will work — typically those that do pre-processing, and others will need to be modified — typically those that do post-processing at the end of a request. Such filters will need to recognize when the initial Servlet container thread is being exited, making way for another thread to continue processing, and when they are invoked as part of an asynchronous dispatch to complete processing.

The OpenSessionInViewFilter and OpenEntityManagerInViewFilter have been updated to work transparently over the span the entire async request. However, if using @Transactional directly on a controller method, the transaction will complete as soon as the controller method returns and will not extend to the execution of the Callable. If the Callable needs to do transactional work it should delegate to a bean with @Transactional methods.

The next post explores the use of DeferredResult for async processing by modifying an existing sample from the Spring AMQP project that reacts to AMQP messages and sends updates to the browser.

Similar Posts

Share this Post
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google Bookmarks
  • DZone
  • LinkedIn
  • Slashdot
  • Technorati
  • Twitter
 

41 responses


  1. Thanks for posting. Had the opportunity to tinker with the asyc support. Looks interesting, but it looks like it'll need hand-holding from client side javascript to be really useful.


  2. The Callable helps to scale server-side processing but the client sees a request and response as usual. Can you clarify what you mean by hand-holding from client-side JavaScript?


  3. I was making the same point as you did in your comment. Client side ajax is required to handle the asynchronous behavior.


  4. Very clear explanation about Timeouts, great!

    However, a naïve question Rossen: what happens if I set a timeout value on server-side which exceeds the HTTP connection default timeout ?
    In other sense, what if the underlying TCP connexion expired before the tmeout in AsyncContext is reached ? Will the container be notified of such expiration ? Will the AsyncContext (and its listeners) be notified ? In this case, what will the client see ? A classical HTTP 408 timeout or the result of the timeout management by my AsyncListener (if any) ?


  5. Duyhai, good question! I did some brief experimenting with a short timeout using 'curl –max-time 2' — the client saw a timeout after 2 seconds while the server continued to wait until its own timeout after 10 seconds was reached. I'll have to verify how the container is actually expected to work on that occasion. I've added a comment to SPR-9400. Feel free to comment or place a watch on it.


  6. Hi Rossen,

    Thanks for a great post. I've start playing with the new async support and it seems to work well in most cases. However, when my callable is resolving a view, I'm getting a DispatcherServlet exception.

    Testing is out with the examples in the spring-mvc-async branch, I noticed the same (http://localhost:8080/spring-mvc-showcase/views/html). Is there specific configuration changes I need to make for view resolution to work with async?

    16:58:04 [qtp4697408-12] DispatcherServlet – DispatcherServlet with name 'appServlet' processing GET request for [/spring-mvc-showcase/views/html]
    16:58:04 [qtp4697408-12] RequestMappingHandlerMapping – Looking up handler method for path /views/html
    16:58:04 [qtp4697408-12] RequestMappingHandlerMapping – Returning handler method [public java.util.concurrent.Callable org.springframework.samples.mvc.views.ViewsController.prepare(org.springframework.ui.Model)]
    16:58:04 [qtp4697408-12] DispatcherServlet – Last-Modified value for [/spring-mvc-showcase/views/html] is: -1
    16:58:04 [qtp4697408-12] DispatcherServlet – Exiting request thread and leaving the response open
    16:58:04 [SimpleAsyncTaskExecutor-1] DispatcherServlet – Resuming asynchronous processing of GET request for [/spring-mvc-showcase/views/html]
    16:58:07 [SimpleAsyncTaskExecutor-1] DispatcherServlet – Rendering view [org.springframework.web.servlet.view.JstlView: name 'views/html'; URL [/WEB-INF/views/views/html.jsp]] in DispatcherServlet with name 'appServlet'
    16:58:07 [SimpleAsyncTaskExecutor-1] JstlView – Added model object 'fruit' of type

    [/java] to request in view with name 'views/html'
    16:58:07 [SimpleAsyncTaskExecutor-1] JstlView - Added model object 'foo' of type [java language=".lang.String"]

    to request in view with name 'views/html'
    16:58:07 [SimpleAsyncTaskExecutor-1] DispatcherServlet – Could not complete request
    javax.servlet.ServletException: Could not get RequestDispatcher for [/WEB-INF/views/views/html.jsp]: Check that the corresponding file exists within your web application archive!
    at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:219)
    at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:262)
    at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1265)
    at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1016)
    at org.springframework.web.servlet.DispatcherServlet.access$300(DispatcherServlet.java:141)
    at org.springframework.web.servlet.DispatcherServlet$2.call(DispatcherServlet.java:1054)
    at org.springframework.web.servlet.DispatcherServlet$1.call(DispatcherServlet.java:884)
    at org.springframework.web.servlet.FrameworkServlet$1.call(FrameworkServlet.java:991)
    at org.springframework.web.context.request.async.AsyncExecutionChainRunnable.run(AsyncExecutionChainRunnable.java:64)
    at java.lang.Thread.run(Thread.java:722)
    16:58:07 [SimpleAsyncTaskExecutor-1] AsyncExecutionChainRunnable – Completing async request processing


  7. Kwong, thanks for trying it out. There shouldn't be anything specific you need to do. There is one example with view resolution in the modified version of spring-mvc-showcase.

    A silly question but make sure it work if not returning a Callable. Or if this is something you can reproduce in a small example, feel free to create an issue project and open a ticket in JIRA.


  8. Rossen, thanks for the quick reply.

    The problem seems specific to Jetty (I'm using the last stable Jetty 8.1.3). The same code runs fine on Glassfish v3. Also the non-async version (that returns String instead of Callable) works on both Jetty and Glassfish.

    I've opened a ticket as suggested
    https://jira.springsource.org/browse/SPR-9433


  9. Thanks, Kwong.


  10. Interesting feature which certainly has a great use case. I just want to point a simple wrong spelling:

    "The idea is that all of what you already know about how a controller method works remains unchanged as much as possibble…"

    The word possibble has two "b"s


  11. Thanks, fixed the typo.


  12. please change the org.springframework.http.converter.StringHttpMessageConverter#public static final Charset DEFAULT_CHARSET = Charset.forName("ISO-8859-1") to UTF-8!thanks


  13. Hi Rossen,
    How to configure AsyncRequestTimeout using xml?


  14. Do you mean if using the MVC namespace? It's a property on RequestMappingHandlerAdapter, so just set it in your XML configuration as any other property, or for the MVC namespace see the advanced customizations subsection in the chapter on configuring Spring MVC.


  15. Hi,
    i am a newcomer on Spring MVC.
    I used to realize Async Servlet through startAsync() method of HttpServletRequest(), saving AsyncContext, and using it when a new data is available for the client.
    I saw the StandardServletAsyncWebRequest introduced in Spring 3.2 that exposes a method startAsync() (wrapping the original one).
    How can i use it in Spring MVC? (I already read your tutorial about it, but with no results..)
    Sorry for my English,
    Domenico


  16. Domenico, there are two more blog posts after this one (you'll find links in the last paragraph above). Each post points to a repository with sample code and explains what you should be looking at. In short, to answer you question, in Spring MVC you don't need to call startAsync() or even use something like the AsyncWebRequest. Instead you return a Callable or a DeferredResult from your controller method. The goal is for you to focus on what you need to achieve in your Spring MVC controller, not on the mechanics of the Servlet 3 async API.


  17. Hi, great news to see servlet3.0 async support on spring mvc.. i tried it but unfortunately i'm not lucky :-/

    i don't know why but it's not working for me.

    16:30:40.319 [http-nio-8080-exec-3] – DEBUG c.d.u.DisplayReturnedElementAspect – [token: client:sm user:sm] [cid:1000 uid:1000] [ip: 127.0.0.1] – returned value : org.springframework.web.context.request.async.DeferredResult@7618d289
    16:30:40.320 [http-nio-8080-exec-3] – DEBUG c.xxx.utils.TraceTimerAspect – [token: client:sm user:sm] [cid:1000 uid:1000] [ip: 127.0.0.1] – MessageController.messageAll(..), arg: SecurityContextHolderAwareRequestWrapper[ FirewalledRequest[ org.springframework.web.filter.AbstractRequestLoggingFilter$RequestCachingRequestWrapper@1a996ff]], arg: -1 – duration: 26 ms
    16:30:40.377 [http-nio-8080-exec-3] – WARN c.d.u.DeveryflowExceptionResolver – [token: client:sm user:sm] [cid:1000 uid:1000] [ip: 127.0.0.1] – catched error
    org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation

    my function only contains this code and returns a DeferredResult like your samples

    // request.setTimeout(timeout);
    final DeferredResult result = new DeferredResult();
    final String key = messageDispatcherService.add(broadcasterType, getApi());
    messageDispatcherService.addDeferredResult(key, result);
    return result;


  18. sylvek does the sample work for you? are you using 3.2 M1 or the latest 3.2 snapshot build? which servlet container and version? It may be a good idea to post a message in a forum providing those details and some more of the log output. You can link the forum post here.


  19. Rossen thanks for your response.. It's working now… it's very important to migrate mvc 3.0 to 3.1 configuration before trying your feature..

    in my case.. i use now :

    instead of AnnotationMethodHandlerAdapter .. and it's working better ;-)

    but just a question.. is there a way to manage the timeout programmatically and by request ?


  20. Oops bad copy/paste… i use that now.


  21. bean class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter"
    property name="asyncRequestTimeout" value="60000" /
    property name="messageConverters"
    list
    bean class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter" /
    /list
    /property
    /bean


  22. Great… i found what i need thks :)
    https://fisheye.springsource.org/browse/spring-framework/spring-web/src/main/java/org/springframework/web/context/request/async/DeferredResult.java?hb=true


  23. Yes, there have been some improvements in this area for the upcoming 3.2 M2 release. See SPR-9694 and SPR-9565.


  24. @RequestMapping(value = "/test", method = RequestMethod.GET)
    public ResponseEntity test()
    {
    HttpHeaders headers = new HttpHeaders();
    headers.setContentType(MediaType.TEXT_PLAIN);
    return new ResponseEntity("Hi There!!", headers, HttpStatus.OK);
    }

    works fine. But when I add Callable interface

    @RequestMapping(value = "/test", method = RequestMethod.GET)
    public Callable<ResponseEntity> test()
    {
    return new Callable<ResponseEntity>()
    {
    @Override
    public ResponseEntity call() throws Exception
    {
    Thread.sleep(10000);
    HttpHeaders headers = new HttpHeaders();
    headers.setContentType(MediaType.TEXT_PLAIN);
    return new ResponseEntity("Hi There!!", headers, HttpStatus.OK);
    }
    };
    }

    I get HTTP Status 404 – WARN PageNotFound:1080 – No mapping found for HTTP request with URI [xxxxx] in DispatcherServlet with name 'dispatcher'


  25. Strange indeed, since the mapping did not change. If you can provide a reproducible project, possibly here, please file a ticket.


  26. Hello Rossen,

    Great post. I use Spring 3 mvc in many controllers and the methods all return ModelAndView objects with a ContentNegotiatingView resolver setup. I don't see any examples with this scenario. Does the new 3.2 M2 async support cover this type of setup or should we move off returning ModelAndView's in our controllers for something else? Thanks in advance.


  27. Yes, view resolution should work including a ModelAndView return value, ContentNegotiatingViewResolver, etc. There are definitely examples with view resolution. The Spring MVC Showcase now has a tab for "async" related options.


  28. I got HTTP 404 too
    No mapping found for HTTP request


  29. Mike, are you referring to what Pranay wrote? My guess is that you're not using the RequestMappingHandlerMapping/Adapter added in Spring MVC 3.1.


  30. Hello Rossen,

    I was looking at the samples you pointed me at and I decided to do some playing around. I have the following piece of code that is not too different from your sample.

    @RequestMapping(value = "/**")
    public Callable defaultMethod() {

    return new Callable() {
    @Override
    public String call() throws Exception {
    System.out.println("**************ERROR**************");
    throw new UnsupportedOperationException(
    "Bad Request. The service is not supported.");
    }
    };

    }

    I noticed that in the dispatcher servlet the dispatch exception is never set and thus when processDispatchResult is subsequently called the UnsupportedOperationException never makes it to a custom HandlerExceptionResolver that I have added to the application context. Am I doing something wrong here or am I supposed to handle exceptions thrown in the Callable some other way when using a custom HandlerExceptionResolver?


  31. You're not doing anything wrong, the method looks fine. I presume that async processing does start and the Callable is invoked? Enable DEBUG logging for org.springframework.web. You should see output similar to this:

    [MvcAsync1] WebAsyncManager - Concurrent result value [UnsupportedOperationException: Bad Request. The service is not supported.]
    [MvcAsync1] WebAsyncManager - Dispatching request to resume processing
    [http-bio-8080-exec-1] DispatcherServlet - DispatcherServlet with name '..' resumed processing GET request for [...]
    ...
    [http-bio-8080-exec-1] RequestMappingHandlerAdapter - Found concurrent result value [UnsupportedOperationException: Bad Request. The service is not supported.]
    

    and then the HandlerExceptionResolver should be invoked. I would also verify that the custom HandlerExceptionResolver works if you don't return a Callable.


  32. How do you return custom http headers? Following REST principles, you'd return a Location Header with the FUTURE url for client to poll the status of the Resource so that eventually, it would get a more permanent URL (Request-Acknowledge-Poll pattern).

    Neither returning Callable or MVCAsyncTask support customizing the a HTTPResponse.


  33. There is more than one way to do it. Just take an existing controller method that already does what you want it to do and change it to return a Callable. For example, your method might return a ResponseEntity with a location header. Change it to return a Callable and then return the ResponseEntity from the Callable instead.


  34. Hi Rossen,
    I find this article very informative. thank you for the post. I was wondering if there's a design pattern that you can suggest as to how to implement the base class for all the Controllers that will wrap the anonymous (Callable) class.

    What I am trying to avoid is to have all my methods return an anonymous Callable class.

    Thank you.
    Don


  35. Don, unless I'm missing something, create your own abstract class that implements Callable?


  36. Hey,

    We're big springheads on a new REST springMVC project that is fairly greenfield, so we're trying to spin up an async architecture with elastic cloud, blah blah blah.

    We've selected Jetty for maximum embeddability and speed, and because they seemed to be at the forefront of async support, for testing flexibility, and lots of other reasons. We know tomcat is also embeddable and can do this, but Jetty just seems better.

    Has the springmvc async framework been thoroughly tested with various filters (like a zipping/deflation filter, or other request and response manipulation filters) in Jetty 8 and 9? We're seeing a zip filter that only seems to activate about 10-20% of the time. I'd love to provide you a nice test example, but haven't been able to produce a reliable reproduction. Such is the nature of concurrent programming.

    Any ideas or suspicions? It also appears that this doesn't happen with non-springmvc servlets. Jetty8 running inside eclipse seems to be better than not in eclipse, but we're still kind of thrashing trying to trace down the problem. Jetty9 also shows the problem, but we've broken some stuff swapping dependencies so we can't say authoritatively Jetty9 is busted in the same way.

    It appears s-mvc runs its own threadpool and management for async servlets independent of how the appserver is implementing servlet 3.0 (s-mvc uses a separate response threadpool, but tomcat and jetty seem to use a common threadpool for handling requests and fulfilling responses).


  37. @bbguy, filters fall in various categories. The ones that do only pre-processing will work fine in async scenarios. The ones that do post-processing must be updated to support Servlet 3 async processing, in other words to expect that the initial Servlet container thread will exit without finalizing the response, and then eventually a dispatch will be performed to complete processing in another Servlet container thread.

    We've gone through that process with all the filters in spring-web. In principal any other filters that support Servlet 3 async processing should work too. So make sure those Jetty filters do and let us know. Use JIRA please.

    As for the thread pool, that is entirely expected. The whole idea of Servlet 3 async processing is that you can free the Servlet container thread as soon as possible, do some work in your own thread, and then when done, dispatch back to the container to finalize processing. Furthermore, Spring MVC does use a thread pool when you return a Callable but not when you return a DeferredResult. With a DeferredResult you can use any thread you want. This is explained in this blog post series.


  38. Hi Rossen,

    Thanks. if I create an abstract class, i will end up making a lot of 1 method concrete classes that returns Callable, right?

    In the example above, you have a controller that has a methods that returns an annonymous callable. I'm just trying to think, if we can encapsulate the creation of anonymous callable class, because in this example, every controller method that I want to make asynchronous, will always have a boilerplate code (anonymous Callable) inside it.

    Thanks,
    Don


  39. Hi Rossen,
    Thanks for the writeup.
    Does spring mvc 3.2 support Server-Sent-Events?
    Would be great to have the ability to use the server asynchronicity with a client that supports push notifications.


  40. Hi Joe, Spring MVC 3.2 doesn't but we're working on SockJS support for Spring Framework 4 and one of the SockJS transports is based on Server-Sent events.


  41. Thanks for the reply!
    Sounds interesting.
    Will keep an eye out

4 trackbacks

Leave a Reply