Search This Blog

Friday, June 29, 2012

mvn test: LocalJobRunner.java with java.lang.OutOfMemoryError

When I run mvn test to unit test some hadoop related program, I got the following:
java.lang.OutOfMemoryError: Java heap space
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
I tried to set MAVEN_OPTS=-Xmx2048m, but it didn't work.   After the online research, I fixed this problem by adding the following to pom:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <argLine>-Xms256m -Xmx512m</argLine>
        </configuration>
      </plugin>

Thursday, June 28, 2012

maven shade plugin: Invalid signature file digest for Manifest main attributes

If you get the following error message with maven shade plugin:

Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes

You need to add the following to pom.xml:

        <configuration>
          <filters>
            <filter>
              <artifact>*:*</artifact>
              <excludes>
                <exclude>META-INF/*.SF</exclude>
                <exclude>META-INF/*.DSA</exclude>
                <exclude>META-INF/*.RSA</exclude>
              </excludes>
            </filter>
          </filters>
        </configuration>

Explanation:

The above configuration filters all files in META-INF ending with .SF, .DSA, and .RSA for all artifacts (*:*) when creating uber-jar file.

The reason java.lang.SecurityException is raised is because some dependency jar files are signed jar files.  A jar file is signed by using jarsigner, which creates 2 additional files and places them in META-INF:
  • a signature file, with a .SF extension, and
  • a signature block file, with a .DSA, .RSA, or .EC extension.
Since the uber-jar file is created, the signatures and integrity of signed JAR files are no longer valid.  When the uber-jar file is executed, java.lang.SecurityException is thrown.

See jarsigner for detailed explanation of JAR Signing and Verification Tool.

Sunday, June 24, 2012

Eclipse: fix plugin execution not covered by lifecycle configuration

I recently used maven-jaxb2-plugin to generate java classes based on xml schemas (xsd).  I can run "mvn compile" on command line to generate java classes with the following configuration:
            <plugin>
                <groupId>org.jvnet.jaxb2.maven2</groupId>
                <artifactId>maven-jaxb2-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>generate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

However, my Eclipse shows the following error:

Plugin execution not covered by lifecycle configuration: org.jvnet.jaxb2.maven2:maven-jaxb2-plugin:0.8.1:generate (execution: default, phase: generate-sources)


This error prevents Eclipse generating java source codes and adding the generated sources to the class path.  In order to fix this problem, I need to install m2e connector for maven-jaxb2-plugin.  I can manually install it to Eclipse with the following update site:

http://bitstrings.github.com/m2e-connectors-p2/milestones/

However, Eclipse provides a better to install it.  Eclipse shows the problem with the following:


Roll the mouse over <execution>, it displays the following:
Click the link "Discover new m2e connector", and eclipse will try to find the correct  connector automatically:
Eclipse found the same m2e connector.  Click finish to install it.  After I installed it successfully, Eclipse is happy with maven-jaxb2-plugin.

This approach can be applied to other plugins too.

Friday, June 8, 2012

Enable CORS support in REST services with Spring 3.1

We are working on a web project, which has 2 components, UI and web services.  These 2 components are deployed to 2 different servers.  UI uses jQuery to make ajax call  to back end restful web services. Back end web services use Spring 3.1 MVC framework.  We are facing the XSS (cross site scripting) issue since modern browsers don't allow XSS.

The back end rest web services support HTTP methods GET, POST, PUT and DELETE.  The first problem we solved was how to call HTTP method GET from UI using jQuery.  Since I used JSONP before, it is a natural solution to me to use JSONP again.  My coworker already blog the solution here: http://www.iceycake.com/2012/06/xml-json-jsonp-web-service-endpoints-spring-3-1.

However,  JSONP only supports GET, and you can't set http headers with JSONP request.  I also realized that JSONP is kind of out-of-dated too.  So how to support other methods like PUT, POST? 

CORS, Cross-Origin Resource Sharing, defines a mechanism to enable client-side cross-origin requests.  CORS are widely supported by modern browsers like FireFox, Chrome, Safari, and IE.

The UI code is fairly simple:
<html>
<head>
<script type="text/javascript" src="jquery.js"></script>
<script type="application/javascript">      
(function($) {
    var url = 'http://localhost:8080/employee/id';
    $.ajax({
        type: 'put',
        url: url,
        async: true,
    contentType: 'application/json',
        data: '{"id": 1, "name": "John Doe"}',
        success: function(response) {
            alert("success");
        },
        error: function(xhr) {
            alert('Error!  Status = ' + xhr.status + " Message = " + xhr.statusText);
        }
    });
})(jQuery);
</script>
</head>
<body>
    <!-- we will add our HTML content here -->
</body>
</html>
If you just run the above javascript and call the backend, you may notice that the PUT request is changed to OPTIONS.  The browser sends an OPTIONS request to the server, and the server needs to send response with correct headers.  So in order to return the correct response, a filter is created (thanks to the link https://gist.github.com/2232095):
package com.zhentao;
import java.io.IOException;
import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.springframework.web.filter.OncePerRequestFilter;
public class CorsFilter extends OncePerRequestFilter {

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
            throws ServletException, IOException {
        if (request.getHeader("Access-Control-Request-Method") != null && "OPTIONS".equals(request.getMethod())) {
            // CORS "pre-flight" request
            response.addHeader("Access-Control-Allow-Origin", "*");
            response.addHeader("Access-Control-Allow-Methods", "GET, POST, PUT, DELETE");
            response.addHeader("Access-Control-Allow-Headers", "Content-Type");
            response.addHeader("Access-Control-Max-Age", "1800");//30 min
        }
        filterChain.doFilter(request, response);
    }
}
 The web.xml needs adding the following too:
  <filter>
    <filter-name>cors</filter-name>
    <filter-class>com.zhentao.CorsFilter</filter-class>
  </filter>
  <filter-mapping>
    <filter-name>cors</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

   
However, this isn't enough yet.  The original post didn't address it.  You still need to change your rest web services to return Accept-Controll-Allow-Origin header since HTTP is stateless.  Here is what I did with Spring Rest api:
    @RequestMapping(value = "/employee/{id}", method = RequestMethod.PUT, consumes = {"application/json"})
    public ResponseEntity<Employee> create(@Valid @RequestBody Employee employee, @PathVariable String id) 
            employee.setId(id);//some logic here
            HttpHeaders headers = new HttpHeaders();
            headers.add("Access-Control-Allow-Origin", "*");
            ResponseEntity<Employee> entity = new ResponseEntity<Employee>(headers, HttpStatus.CREATED);
            return entity;
    }
Update: it seems my post caused some confusion, and I created another post and fully working examples for CORS. See here.