Search This Blog

Friday, June 29, 2012

mvn test: LocalJobRunner.java with java.lang.OutOfMemoryError

When I run mvn test to unit test some hadoop related program, I got the following:
java.lang.OutOfMemoryError: Java heap space
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:428)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
I tried to set MAVEN_OPTS=-Xmx2048m, but it didn't work.   After the online research, I fixed this problem by adding the following to pom:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <argLine>-Xms256m -Xmx512m</argLine>
        </configuration>
      </plugin>

No comments:

Post a Comment