How to download JavaDocs or sources in Maven 2?

Problem

You want to download javadocs for your depenencies into local repository to be able to use them ie. in IntelliJ.

Solution


mvn dependency:resolve -Dclassifier=javadoc
mvn dependency:resolve -Dclassifier=sources


Enjoy!

ANT script good practices – few thoughts

In general scripts should be treated as first class citizens, so they need to be simple, modular, clean, easy to read and understand and loosely coupled.

Here are some techniques which should help with it:

All the OO principles apply!

When you need custom functionality for your project i.e. custom make step – do not bloat the common files with conditional logic. Extract common stuff to macrodefs and create separate files and/or targets for custom functionality in local project directory. Basically all the OO principles apply!

Break it down and modularize

You wouldn’t like to have just one big class in your project – do the same with ant script, break it down and modularize

Stop writing generic ant targets parametrized by global properties

Replace them with macrodefs. See:
http://ant.apache.org/manual/CoreTasks/macrodef.html

Stop using global properties

Don’t use them, unless they are really global i.e. project.name, project.version. In general they are evil as global variables and singletons are. See:
http://blogs.msdn.com/scottdensmore/archive/2004/05/25/140827.aspx
http://www.cs.usfca.edu/~wolber/courses/110/lectures/globals.htm

Use convention over configuration design paradigm.

What does it mean? Basically this:
Instead of writing (unneccessary abstraction and complexity):

<property name="project.dir" value="${basedir}"/>
<property name="test.result.dir" value="${project.dir}/build/testresult"/>
<javadoc dest="${test.result.dir}>
...


We would write:

<javadoc dest="build/testresult">
...

Simpler and more readable!
See more here:
http://en.wikipedia.org/wiki/Convention_over_Configuration

Use maven-like project directory structure

This supports convention over configuration approach and simplifies your build scripts.

project
- src
  +- main
     +- java
     +- config
     +- scripts
     +- ...
  +- test
     +- java
     +- config
     +- ...
- lib
  +- compile
  +- runtime
     +- spring-2.0.jar
     +- jakarta-commons
        +- commons-io.jar
        +- commons-???.jar
     +- ...
  +- test
- ant (contains common ant tasks - it should be symbolic link as this scripts should be shared)
  +- ant-common.xml
  +- ant-test.xml
  +- ...
- target ( temporary directory, only RW area of the project, equivalent to build)
- modules (if the code contains modules build separately like c++)
  +- SomeNativeLib
      +- src
      +- lib
      +- ...
  +- OtherJavaModule
      +- src
      +- lib
      +- ...
- build.xml

Enjoy!

Mocking difficult objects like HttpServletRequest with CGLIB

Why haven’t I seen this before?
http://www.sixlegs.com/blog/java/concrete-mock-objects.html

Encoding in forms sent using POST method

I struggled a lot with encoding in Tomcat before I found this post on Adam’s Warski blog.

How to UnZip archive in Java

Why am I writing about it?

Because I could not find complete solution for unzipping archive in java.

Dependencies

You will need commons-io.jar from apache commons.

Random thoughts…

It is too small to release it as an open source library, and Apache Commons does not contain such a function, when I am writing it this post. Please feel free to copy/use/modify this code.

public class ZipUtils {
    Logger log;

    public ZipUtils(Logger log) {
        this.log = log;
    }

    public void unzipArchive(File archive, File outputDir) {
        try {
            ZipFile zipfile = new ZipFile(archive);
            for (Enumeration e = zipfile.entries(); e.hasMoreElements(); ) {
                ZipEntry entry = (ZipEntry) e.nextElement();
                unzipEntry(zipfile, entry, outputDir);
            }
        } catch (Exception e) {
            log.error("Error while extracting file " + archive, e);
        }
    }

    private void unzipEntry(ZipFile zipfile, ZipEntry entry, File outputDir) throws IOException {

        if (entry.isDirectory()) {
            createDir(new File(outputDir, entry.getName()));
            return;
        }

        File outputFile = new File(outputDir, entry.getName());
        if (!outputFile.getParentFile().exists()){
            createDir(outputFile.getParentFile());
        }

        log.debug("Extracting: " + entry);
        BufferedInputStream inputStream = new BufferedInputStream(zipfile.getInputStream(entry));
        BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream(outputFile));

        try {
            IOUtils.copy(inputStream, outputStream);
        } finally {
            outputStream.close();
            inputStream.close();
        }
    }

    private void createDir(File dir) {
        log.debug("Creating dir "+dir.getName());
        if(!dir.mkdirs()) throw new RuntimeException("Can not create dir "+dir);
    }
}

Infinitest and IntelliJ

Infinitest is a very interesting tool for continuous testing. I am looking forward for IntelliJ plugin.

In the mean time
Here is how you can run it in IntelliJ now:

1. Run your tests as usual from IntelliJ
2. Go to console window and copy -classpath x;y;z…
3. Create application run configuration for class org.infinitest.Infinitest as usual and add the copied string to VM parameters field.
4. Run newly created configuration

Enjoy!

Aspect Oriented Programming – real life examples

Some time ago, on my previous blog, I wrote a short series of articles showing real live examples of using AOP. It is time to group them together and share with you again:

• Introduction and caching example Transparent Caching with AspectJ
• Tracing history of bean with AspectJ in 10 minutes
• Retry if failed pattern with AspectJ in 10 minutes
• @Configurable without Spring in 10 minutes

Enjoy!

RmiCluster – working alternative for cluster4spring

Business problem

I recently spent couple of weeks on writing a distributed Market Data Service for my client. The requirement was to distribute the load on couple of machines as MDS was meant to be used by few hundred node grid.

Cluster4Spring

We investigated couple of options and decided to go for Cluster4Spring. It gave us load distribution, failover and dynamic discovery.

Failed under load

After couple of days of testing it turned out that we needed to switch the dynamic discovery off as some client nodes were failing to find MDS. This left us with predefined list of RMI endpoints. Unfortunately few days later we realised that some of the nodes were loosing connections to all MDS instances in the middle of the job. I ruled out the network problems. MDS instances were not overloaded, so they were not dropping connections. BTW it struck me that a single node could only handle 50 requests per second. Internal processing of request was taking no more than 1ms ( return map.get(key) ). I tried different config options but nothing helped.

RmiCluster

At this point I was desperate enough to write my own clustering proxy to replace Cluster4Spring. When I thought about it, it looked like simple task to do. You maintain a list of alive nodes, refresh it periodically. If invocation fails remove node from the list. When invoking you just pick random remote node and that is it.

It took me 150 lines, and hey, it works! Single node is able to handle in excess of 100 requests per second.

To use it just expose your beans using spring RmiServiceExporter on the server side:

  1     <bean class="org.springframework.remoting.rmi.RmiServiceExporter">
  2         <property name="serviceName" value="serviceX"/>
  3         <property name="service" ref="MarketDataService"/>
  4         <property name="serviceInterface" value="com.XXX.marketdata.server.MarketDataServer"/>
  5         <property name="registry" ref="RMIRegistry"/>
  6     </bean>

On the client side it could look like this:

  1     <bean name="MarketDataService" class="org.javaexpert.rmicluster.ClusteringRmiProxyFactoryBean">
  2         <property name="serviceInterface" value="com.XXX.marketdata.server.MarketDataServer"/>
  3         <property name="serviceUrl" value="whatever"/>
  4         <property name="serviceUrls">
  5             <list>
  6                 <value>rmi://somehost1:1234/ServiceX</value>
  7                 <value>rmi://somehost2:1234/ServiceX</value>
  8                 <value>rmi://somehost3:1234/ServiceX</value>
  9                 <value>rmi://somehost4:1234/ServiceX</value>
 10             </list>
 11         </property>
 12         <property name="lookupStubOnStartup" value="true"/>
 13     </bean>
 14 

ClusteringRmiProxyFactoryBean:

  1 package org.javaexpert.rmicluster;
  2
  3 import java.net.MalformedURLException;
  4 import java.rmi.NotBoundException;
  5 import java.rmi.Remote;
  6 import java.rmi.RemoteException;
  7 import java.rmi.registry.LocateRegistry;
  8 import java.rmi.registry.Registry;
  9 import java.util.ArrayList;
 10 import java.util.HashMap;
 11 import java.util.List;
 12 import java.util.Map;
 13 import java.util.Random;
 14 import java.util.Timer;
 15 import java.util.TimerTask;
 16 import java.util.regex.Matcher;
 17 import java.util.regex.Pattern;
 18
 19 import org.aopalliance.intercept.MethodInvocation;
 20 import org.javaexpert.error.ErrorUtils;
 21 import org.springframework.beans.factory.DisposableBean;
 22 import org.springframework.remoting.RemoteAccessException;
 23 import org.springframework.remoting.RemoteLookupFailureException;
 24 import org.springframework.remoting.rmi.RmiProxyFactoryBean;
 25
 26
 27 public class ClusteringRmiProxyFactoryBean extends RmiProxyFactoryBean implements DisposableBean{
 28     // Inputs
 29     private List<String> serviceUrls;
 30     private int refreshEndpointsMillis = 60*1000;
 31
 32     // Internals
 33     private Map<String,Remote> aliveServices = new HashMap<String, Remote>();
 34     private Map<Remote, String> aliveServicesReverseMap = new HashMap<Remote, String>();
 35     private Random rand = new Random(System.currentTimeMillis());
 36     private Pattern rmiAddressPattern = Pattern.compile("rmi://(.+):([0-9]*)/(.+)");
 37     private Timer timer;
 38
 39     protected synchronized Remote lookupStub() throws RemoteLookupFailureException{
 40         refreshServicesIfNeededUntilFoundOne();
 41         return new ArrayList<Remote>(aliveServices.values()).get(rand.nextInt(aliveServices.size()));
 42     }
 43
 44     protected Remote getStub() throws RemoteLookupFailureException{
 45         return lookupStub();
 46     }
 47
 48     private synchronized void refreshServicesIfNeededUntilFoundOne(){
 49         while(aliveServices.size() == 0){
 50             logger.info("No services alive - refreshing endpoints.");
 51             refreshServices();
 52         }
 53     }
 54
 55     public void refreshServices(){
 56         for(String url: serviceUrls){
 57             if (!aliveServices.containsKey(url)){
 58                 try{
 59                     Remote remote = createRemote(url);
 60                     addServiceToAlive(url, remote);
 61                 } catch(Exception e){
 62                     logger.warn("Can not connect to " + url+ " - ignoring.");
 63                 }
 64             }
 65         }
 66     }
 67
 68     public Remote createRemote(String url) throws RemoteException, NotBoundException, MalformedURLException{
 69         Matcher res = rmiAddressPattern.matcher(url);
 70         if (!res.matches()) throw new MalformedURLException("Wrong address syntax - '" + url +"'. Correct syntax is rmi://host:port/service");
 71
 72         Registry reg = LocateRegistry.getRegistry(res.group(1), Integer.valueOf(res.group(2)));
 73         return reg.lookup(res.group(3));
 74     }
 75
 76     public Object invoke(final MethodInvocation invocation) throws Throwable{
 77         return ErrorUtils.executeWithRetry(100, "Invocation failed - retrying...", 5000, new ErrorUtils.CallableWithRecovery(){
 78             private Remote stub;
 79             public void recover() throws Throwable{
 80                 removeServiceFromAlive(stub);
 81             }
 82
 83             public Object call() throws Throwable{
 84                 refreshServicesIfNeededUntilFoundOne();
 85                 stub = getStub();
 86                 return doInvoke(invocation, stub);
 87             }
 88         }, RemoteAccessException.class);
 89     }
 90
 91     private synchronized void addServiceToAlive(String url, Remote remote){
 92         aliveServices.put(url, remote);
 93         aliveServicesReverseMap.put(remote, url);
 94     }
 95
 96     private synchronized void removeServiceFromAlive(Remote s){
 97         String url = aliveServicesReverseMap.remove(s);
 98         aliveServices.remove(url);
 99     }
100
101     public void setServiceUrls(List<String> serviceUrls){
102         this.serviceUrls = serviceUrls;
103     }
104
105     public void afterPropertiesSet(){
106         setRefreshStubOnConnectFailure(false);
107         setCacheStub(false);
108         setServiceUrl("legacy");
109         super.afterPropertiesSet();
110
111         initRefreshTimer();
112     }
113
114     private void initRefreshTimer(){
115         timer = new Timer();
116         timer.schedule(new TimerTask(){
117             public void run(){
118                 try{
119                     refreshServices();
120                 } catch(Exception e){
121                     logger.warn("Can not refresh services", e);
122                 }
123             }
124         }, refreshEndpointsMillis, refreshEndpointsMillis);
125     }
126
127     public void destroy() throws Exception{
128         timer.cancel();
129     }
130
131     public void setRefreshEndpointsMillis(int refreshEndpointsMillis){
132         this.refreshEndpointsMillis = refreshEndpointsMillis;
133     }
134 }

Here is ErrorUtils:

 1 package org.javaexpert.error;
 2
 3 import java.util.Set;
 4
 5
 6 import java.util.Arrays;
 7 import java.util.HashSet;
 8 import java.util.logging.Logger;
 9
10
11 public class ErrorUtils {
12     private static final Logger LOGGER = Logger.getLogger(ErrorUtils.class.getCanonicalName());
13
14     public static <T> T executeWithRetry(int retryCount, String errorMessage, int sleepMillis, Callable<T> callable, Class... dontIgnoreExceptions) throws Throwable {
15         Set dontIgnoreSet = convertToSet(dontIgnoreExceptions);
16         Throwable lastException = null;
17         while (retryCount-- > 0) {
18             try {
19                 return callable.call();
20             } catch (Throwable e) {
21                 if (!dontIgnoreSet.isEmpty() && !contains(dontIgnoreSet, e)) throw e;
22                 lastException = e;
23                 LOGGER.warning(errorMessage + " Cause: " + e.getMessage());
24                 LOGGER.info("Sleeping for " + sleepMillis + " millis");
25                 sleep(sleepMillis);
26                 LOGGER.info("Retrying...");
27                 if (callable instanceof CallableWithRecovery) {
28                     try {
29                         ((CallableWithRecovery) callable).recover();
30                     } catch (Throwable throwable) {
31                         LOGGER.warning("Recovery action failed.");
32                     }
33                 }
34             }
35         }
36         throw lastException;
37     }
38
39     private static boolean contains(Set<Class> dontIgnoreSet, Throwable e){
40         for(Class c: dontIgnoreSet){
41             if (c.isInstance(e)) return true;
42         }
43         return false;
44
45     }
46
47     private static Set convertToSet(Object... objects){
48         Set res = new HashSet();
49         res.addAll(Arrays.asList(objects));
50         return res;
51     }
52
53     private static void sleep(int sleepMillis) {
54         try {
55             Thread.sleep(sleepMillis);
56         } catch (InterruptedException e1) {
57             LOGGER.severe("Interrupted"+ e1);
58             throw new RuntimeException(e1);
59         }
60     }
61
62     public interface Callable<T> {
63         T call() throws Throwable;
64     }
65
66     public interface CallableWithRecovery extends Callable {
67         void recover() throws Throwable;
68     }
69 }

IntelliJ idea.exe.vmoptions revised for version 7 and JDK 6

IntelliJ 7 was a bit unstable with JDK 1.6 on previous set of options I presented on this blog so I decided to tweak it again.

Here are the options I ended up with:

-Xms256m
-Xmx256m
-XX:MaxPermSize=150m
-XX:MaxGCPauseMillis=10
–XX:MaxHeapFreeRatio=70
-XX:+UseConcMarkSweepGC
–XX:+CMSIncrementalPacing
-Didea.no.jdk.check=true

Wait-free, lock-free in Java 5 concurrent

I recently came across few interesting articles by Brian Goetz:

• More flexible, scalable locking in JDK 5.0
• Going atomic
• Introduction to nonblocking algorithms
• Readers/Writers problem when using Map
• Wikipedia article

I recommend it to anyone interested in parallel programming in Java!