Executing COMTOR as a stand-alone application

Often times, it is easier to execute COMTOR as a stand-along application, rather than to do so as a web-based application. This is particularly true if you are attempting to debug a module, or write your own.

The stand-alone version of COMTOR does not utilize the back-end database, so all execution is performed locally and no results are stored beyond a text-based report output file. Additionally, you can execute COMTOR on any machine that can execute Java. Depending on the analysis module(s) executed, the stand alone application may (or may not) require the use of the Internet to obtain supporting files. One such example of this is are the supporting files downloaded from COMTOR.org when the spell check modules execute.

Building the Jar File
To create the stand-alone version of COMTOR, first download the source code base via GIT from Sourceforge.net. SmartGit is an excellent Java-based Git client, but requires the git command line tools installed on your system for access (usually not part of a base operating system installation).

Once the source code base has been downloaded, navigate to the code directory and initiate the build process by issuing the following Ant command.

ant makeStandAlone

You should see the following output as the code is compiled and and executable jar file is created for you:

Buildfile: /Users/depasqua/Desktop/TCNJ/comtor/code/build.xml

init: [mkdir] Created dir: /Users/depasqua/Desktop/TCNJ/comtor/code/classes

compile: [javac] Compiling 25 source files to /Users/depasqua/Desktop/TCNJ/comtor/code/classes [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details.

makeStandAlone: [jar] Building jar: /Users/depasqua/Desktop/TCNJ/comtor/code/comtor.jar

BUILD SUCCESSFUL Total time: 13 seconds

Once you have a jar file, copy it to another location from where you plan to run the application and analyze your code.

Creating the properties file
Before we run the application, we need to create a docletList.properties file which will define the list of COMTOR analyzers to execute. This file is a plain-text (non-XML) Java Properties file (see java.lang.String) the API) use to create the list of doclets to execute. The basic format of the properties file is a series of lines of name/value pairs, where the names are the string(s): doclet1, doclet2, etc., and the values are the fully qualified classnames of the COMTOR analysis modules to execute. For example: doclet1 : comtor.analyzers.OffensiveWords doclet2 : comtor.analyzers.SpellCheck

The file contents are read by the stand-alone application at start time, and added to the list of analysis modules to execute on the specified code base. At least one line of input must be present for the client to execute. There is no known limit as to the number of modules one can execute or that the file may contain.

The docletList.properties file should be placed in the same directory location as the comtor.jar file created in the previous step.

Executing the Client
Now that we have the jar file and the list of modules to execute, we can execute the stand-alone application on a code base. Assume for the purposes of this tutorial that also contained in the working directory (that also contains the jar file and properties file) is a subdirectory named foo which contains a code base of Java source code. This foo directory may or may not be a Java package directory for the underlying code.

The basic comment to execute the comtor.jar file is:

java -jar comtor.jar

However, we need to pass a few options to the program to inform it how to execute. First, we need to provide one or more directory paths (similar to how the Java CLASSPATH environment variable operates) to point the tool at the code base. The --sourcepath option allows for this capability. For example:

java -jar comtor.jar -dir files/ant-1.8.2/apache-ant-1.8.2/src/main

In the example above, we tell the COMTOR application to begin searching for Java source code and package structures in the files/ant-1.8.2/apache-ant-1.8.2/src/main directory. Contained within this directory should be the start of the package structure. In this case, the Apache Ant project code is packaged as follows.

org.apache.tools.ant org.apache.tools.ant.dispatch org.apache.tools.ant.filters org.apache.tools.ant.filters.util org.apache.tools.ant.helper org.apache.tools.ant.input org.apache.tools.ant.launch org.apache.tools.ant.listener org.apache.tools.ant.loader org.apache.tools.ant.property org.apache.tools.ant.taskdefs org.apache.tools.ant.taskdefs.compilers org.apache.tools.ant.taskdefs.condition org.apache.tools.ant.taskdefs.cvslib org.apache.tools.ant.taskdefs.email org.apache.tools.ant.taskdefs.optional org.apache.tools.ant.taskdefs.optional.ccm org.apache.tools.ant.taskdefs.optional.clearcase org.apache.tools.ant.taskdefs.optional.depend org.apache.tools.ant.taskdefs.optional.depend.constantpool org.apache.tools.ant.taskdefs.optional.ejb org.apache.tools.ant.taskdefs.optional.extension org.apache.tools.ant.taskdefs.optional.extension.resolvers org.apache.tools.ant.taskdefs.optional.i18n org.apache.tools.ant.taskdefs.optional.image org.apache.tools.ant.taskdefs.optional.j2ee org.apache.tools.ant.taskdefs.optional.javacc org.apache.tools.ant.taskdefs.optional.javah org.apache.tools.ant.taskdefs.optional.jdepend org.apache.tools.ant.taskdefs.optional.jlink org.apache.tools.ant.taskdefs.optional.jsp org.apache.tools.ant.taskdefs.optional.jsp.compilers org.apache.tools.ant.taskdefs.optional.junit org.apache.tools.ant.taskdefs.optional.native2ascii org.apache.tools.ant.taskdefs.optional.net org.apache.tools.ant.taskdefs.optional.perforce org.apache.tools.ant.taskdefs.optional.pvcs org.apache.tools.ant.taskdefs.optional.script org.apache.tools.ant.taskdefs.optional.sos org.apache.tools.ant.taskdefs.optional.sound org.apache.tools.ant.taskdefs.optional.splash org.apache.tools.ant.taskdefs.optional.ssh org.apache.tools.ant.taskdefs.optional.testing org.apache.tools.ant.taskdefs.optional.unix org.apache.tools.ant.taskdefs.optional.vss org.apache.tools.ant.taskdefs.optional.windows org.apache.tools.ant.taskdefs.rmic org.apache.tools.ant.types org.apache.tools.ant.types.mappers org.apache.tools.ant.types.optional org.apache.tools.ant.types.optional.depend org.apache.tools.ant.types.optional.image org.apache.tools.ant.types.resolver org.apache.tools.ant.types.resources org.apache.tools.ant.types.resources.comparators org.apache.tools.ant.types.resources.selectors org.apache.tools.ant.types.selectors org.apache.tools.ant.types.selectors.modifiedselector org.apache.tools.ant.types.spi org.apache.tools.ant.util org.apache.tools.ant.util.depend org.apache.tools.ant.util.depend.bcel org.apache.tools.ant.util.facade org.apache.tools.ant.util.java15 org.apache.tools.ant.util.optional org.apache.tools.ant.util.regexp org.apache.tools.bzip2 org.apache.tools.mail org.apache.tools.tar org.apache.tools.zip

Each of the above listed packages was found through file searching by the COMTOR application starting in the provided directory (files/ant-1.8.2/apache-ant-1.8.2/src/main</tt>). The application will traverse all subdirectories looking for .java files. When any directories are found that contain .java files, they are added to a list which will be the basis of input set for the analysis modules.

When a subdirectory is located that contains .java files, its full path is stored in a list. That is, the org.apache.tools.zip</tt> package is really located in the files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/zip</tt> directory. This path however, needs to be turned into a Java package name (org.apache.tools.zip</tt>) and passed to the Javadoc program for processing. In the final step, we convert the path separator characters (/</tt> in this example) into periods (.</tt>) to form the complete package name: org.apache.tools.zip</tt>.

Summary
After initiating the execution of COMTOR as above, the following output will be provided on the System.out</tt> stream.
 * The list of "found" unpackaged code.
 * The list of "found" packages.
 * The options passed to the Javadoc program (as if it were executed on the command line).
 * Confirmation of the processing of source code files within Javadoc.
 * Any compilation errors that may be present. These are not show-stoppers, but usually are of the form of unresolved symbols due to missing code. See the example below of the processing of the Apache Ant code base. There are unresolved symbols, but Javadoc is only informing you that the symbols are not found, and continues its processing.
 * Any compile-time warnings (such as tag issues, as shown below).

In addition to the output above provided to the System.out/err</tt> stream, a comtorReport.txt file is also created if the execution is successful. Since it can be of a considerable size, an example output file is provided via [[Media:ComtorReport.txt | file link]] rather than embedded into this page.

System.out Example Output
Note: the following output same has been reduced for brevity. Unpackaged source files found in dir: apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/

Found the following packages: org.apache.tools.ant org.apache.tools.ant.dispatch org.apache.tools.ant.filters org.apache.tools.ant.filters.util org.apache.tools.ant.helper org.apache.tools.ant.input org.apache.tools.ant.launch org.apache.tools.ant.listener org.apache.tools.ant.loader org.apache.tools.ant.property org.apache.tools.ant.taskdefs org.apache.tools.ant.taskdefs.compilers org.apache.tools.ant.taskdefs.condition org.apache.tools.ant.taskdefs.cvslib org.apache.tools.ant.taskdefs.email org.apache.tools.ant.taskdefs.optional org.apache.tools.ant.taskdefs.optional.ccm org.apache.tools.ant.taskdefs.optional.clearcase org.apache.tools.ant.taskdefs.optional.depend org.apache.tools.ant.taskdefs.optional.depend.constantpool org.apache.tools.ant.taskdefs.optional.ejb org.apache.tools.ant.taskdefs.optional.extension org.apache.tools.ant.taskdefs.optional.extension.resolvers org.apache.tools.ant.taskdefs.optional.i18n org.apache.tools.ant.taskdefs.optional.image org.apache.tools.ant.taskdefs.optional.j2ee org.apache.tools.ant.taskdefs.optional.javacc org.apache.tools.ant.taskdefs.optional.javah org.apache.tools.ant.taskdefs.optional.jdepend org.apache.tools.ant.taskdefs.optional.jlink org.apache.tools.ant.taskdefs.optional.jsp org.apache.tools.ant.taskdefs.optional.jsp.compilers org.apache.tools.ant.taskdefs.optional.junit org.apache.tools.ant.taskdefs.optional.native2ascii org.apache.tools.ant.taskdefs.optional.net org.apache.tools.ant.taskdefs.optional.perforce org.apache.tools.ant.taskdefs.optional.pvcs org.apache.tools.ant.taskdefs.optional.script org.apache.tools.ant.taskdefs.optional.sos org.apache.tools.ant.taskdefs.optional.sound org.apache.tools.ant.taskdefs.optional.splash org.apache.tools.ant.taskdefs.optional.ssh org.apache.tools.ant.taskdefs.optional.testing org.apache.tools.ant.taskdefs.optional.unix org.apache.tools.ant.taskdefs.optional.vss org.apache.tools.ant.taskdefs.optional.windows org.apache.tools.ant.taskdefs.rmic org.apache.tools.ant.types org.apache.tools.ant.types.mappers org.apache.tools.ant.types.optional org.apache.tools.ant.types.optional.depend org.apache.tools.ant.types.optional.image org.apache.tools.ant.types.resolver org.apache.tools.ant.types.resources org.apache.tools.ant.types.resources.comparators org.apache.tools.ant.types.resources.selectors org.apache.tools.ant.types.selectors org.apache.tools.ant.types.selectors.modifiedselector org.apache.tools.ant.types.spi org.apache.tools.ant.util org.apache.tools.ant.util.depend org.apache.tools.ant.util.depend.bcel org.apache.tools.ant.util.facade org.apache.tools.ant.util.java15 org.apache.tools.ant.util.optional org.apache.tools.ant.util.regexp org.apache.tools.bzip2 org.apache.tools.mail org.apache.tools.tar org.apache.tools.zip

javadoc options: org.apache.tools.ant org.apache.tools.ant.dispatch org.apache.tools.ant.filters org.apache.tools.ant.filters.util org.apache.tools.ant.helper org.apache.tools.ant.input org.apache.tools.ant.launch org.apache.tools.ant.listener org.apache.tools.ant.loader org.apache.tools.ant.property org.apache.tools.ant.taskdefs org.apache.tools.ant.taskdefs.compilers org.apache.tools.ant.taskdefs.condition org.apache.tools.ant.taskdefs.cvslib org.apache.tools.ant.taskdefs.email org.apache.tools.ant.taskdefs.optional org.apache.tools.ant.taskdefs.optional.ccm org.apache.tools.ant.taskdefs.optional.clearcase org.apache.tools.ant.taskdefs.optional.depend org.apache.tools.ant.taskdefs.optional.depend.constantpool org.apache.tools.ant.taskdefs.optional.ejb org.apache.tools.ant.taskdefs.optional.extension org.apache.tools.ant.taskdefs.optional.extension.resolvers org.apache.tools.ant.taskdefs.optional.i18n org.apache.tools.ant.taskdefs.optional.image org.apache.tools.ant.taskdefs.optional.j2ee org.apache.tools.ant.taskdefs.optional.javacc org.apache.tools.ant.taskdefs.optional.javah org.apache.tools.ant.taskdefs.optional.jdepend org.apache.tools.ant.taskdefs.optional.jlink org.apache.tools.ant.taskdefs.optional.jsp org.apache.tools.ant.taskdefs.optional.jsp.compilers org.apache.tools.ant.taskdefs.optional.junit org.apache.tools.ant.taskdefs.optional.native2ascii org.apache.tools.ant.taskdefs.optional.net org.apache.tools.ant.taskdefs.optional.perforce org.apache.tools.ant.taskdefs.optional.pvcs org.apache.tools.ant.taskdefs.optional.script org.apache.tools.ant.taskdefs.optional.sos org.apache.tools.ant.taskdefs.optional.sound org.apache.tools.ant.taskdefs.optional.splash org.apache.tools.ant.taskdefs.optional.ssh org.apache.tools.ant.taskdefs.optional.testing org.apache.tools.ant.taskdefs.optional.unix org.apache.tools.ant.taskdefs.optional.vss org.apache.tools.ant.taskdefs.optional.windows org.apache.tools.ant.taskdefs.rmic org.apache.tools.ant.types org.apache.tools.ant.types.mappers org.apache.tools.ant.types.optional org.apache.tools.ant.types.optional.depend org.apache.tools.ant.types.optional.image org.apache.tools.ant.types.resolver org.apache.tools.ant.types.resources org.apache.tools.ant.types.resources.comparators org.apache.tools.ant.types.resources.selectors org.apache.tools.ant.types.selectors org.apache.tools.ant.types.selectors.modifiedselector org.apache.tools.ant.types.spi org.apache.tools.ant.util org.apache.tools.ant.util.depend org.apache.tools.ant.util.depend.bcel org.apache.tools.ant.util.facade org.apache.tools.ant.util.java15 org.apache.tools.ant.util.optional org.apache.tools.ant.util.regexp org.apache.tools.bzip2 org.apache.tools.mail org.apache.tools.tar org.apache.tools.zip -private -sourcepath apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main Loading source files for package org.apache.tools.ant... Loading source files for package org.apache.tools.ant.dispatch... Loading source files for package org.apache.tools.ant.filters... Loading source files for package org.apache.tools.ant.filters.util... Loading source files for package org.apache.tools.ant.helper... Loading source files for package org.apache.tools.ant.input... Loading source files for package org.apache.tools.ant.launch... Loading source files for package org.apache.tools.ant.listener... Loading source files for package org.apache.tools.ant.loader... Loading source files for package org.apache.tools.ant.property... Loading source files for package org.apache.tools.ant.taskdefs... Loading source files for package org.apache.tools.ant.taskdefs.compilers... Loading source files for package org.apache.tools.ant.taskdefs.condition... Loading source files for package org.apache.tools.ant.taskdefs.cvslib... Loading source files for package org.apache.tools.ant.taskdefs.email... Loading source files for package org.apache.tools.ant.taskdefs.optional... Loading source files for package org.apache.tools.ant.taskdefs.optional.ccm... Loading source files for package org.apache.tools.ant.taskdefs.optional.clearcase... Loading source files for package org.apache.tools.ant.taskdefs.optional.depend... Loading source files for package org.apache.tools.ant.taskdefs.optional.depend.constantpool... Loading source files for package org.apache.tools.ant.taskdefs.optional.ejb... Loading source files for package org.apache.tools.ant.taskdefs.optional.extension... Loading source files for package org.apache.tools.ant.taskdefs.optional.extension.resolvers... Loading source files for package org.apache.tools.ant.taskdefs.optional.i18n... Loading source files for package org.apache.tools.ant.taskdefs.optional.image... Loading source files for package org.apache.tools.ant.taskdefs.optional.j2ee... Loading source files for package org.apache.tools.ant.taskdefs.optional.javacc... Loading source files for package org.apache.tools.ant.taskdefs.optional.javah... Loading source files for package org.apache.tools.ant.taskdefs.optional.jdepend... Loading source files for package org.apache.tools.ant.taskdefs.optional.jlink... Loading source files for package org.apache.tools.ant.taskdefs.optional.jsp... Loading source files for package org.apache.tools.ant.taskdefs.optional.jsp.compilers... Loading source files for package org.apache.tools.ant.taskdefs.optional.junit... Loading source files for package org.apache.tools.ant.taskdefs.optional.native2ascii... Loading source files for package org.apache.tools.ant.taskdefs.optional.net... Loading source files for package org.apache.tools.ant.taskdefs.optional.perforce... Loading source files for package org.apache.tools.ant.taskdefs.optional.pvcs... Loading source files for package org.apache.tools.ant.taskdefs.optional.script... Loading source files for package org.apache.tools.ant.taskdefs.optional.sos... Loading source files for package org.apache.tools.ant.taskdefs.optional.sound... Loading source files for package org.apache.tools.ant.taskdefs.optional.splash... Loading source files for package org.apache.tools.ant.taskdefs.optional.ssh... Loading source files for package org.apache.tools.ant.taskdefs.optional.testing... Loading source files for package org.apache.tools.ant.taskdefs.optional.unix... Loading source files for package org.apache.tools.ant.taskdefs.optional.vss... Loading source files for package org.apache.tools.ant.taskdefs.optional.windows... Loading source files for package org.apache.tools.ant.taskdefs.rmic... Loading source files for package org.apache.tools.ant.types... Loading source files for package org.apache.tools.ant.types.mappers... Loading source files for package org.apache.tools.ant.types.optional... Loading source files for package org.apache.tools.ant.types.optional.depend... Loading source files for package org.apache.tools.ant.types.optional.image... Loading source files for package org.apache.tools.ant.types.resolver... Loading source files for package org.apache.tools.ant.types.resources... Loading source files for package org.apache.tools.ant.types.resources.comparators... Loading source files for package org.apache.tools.ant.types.resources.selectors... Loading source files for package org.apache.tools.ant.types.selectors... Loading source files for package org.apache.tools.ant.types.selectors.modifiedselector... Loading source files for package org.apache.tools.ant.types.spi... Loading source files for package org.apache.tools.ant.util... Loading source files for package org.apache.tools.ant.util.depend... Loading source files for package org.apache.tools.ant.util.depend.bcel... Loading source files for package org.apache.tools.ant.util.facade... Loading source files for package org.apache.tools.ant.util.java15... Loading source files for package org.apache.tools.ant.util.optional... Loading source files for package org.apache.tools.ant.util.regexp... Loading source files for package org.apache.tools.bzip2... Loading source files for package org.apache.tools.mail... Loading source files for package org.apache.tools.tar... Loading source files for package org.apache.tools.zip... Constructing Javadoc information... /Users/depasqua/Desktop/comtor/apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/ant/filters/util/JavaClassHelper.java:22: package org.apache.bcel.classfile does not exist import org.apache.bcel.classfile.ClassParser; ^ /Users/depasqua/Desktop/comtor/apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/ant/filters/util/JavaClassHelper.java:23: package org.apache.bcel.classfile does not exist import org.apache.bcel.classfile.ConstantValue; ^ /Users/depasqua/Desktop/comtor/apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/ant/filters/util/JavaClassHelper.java:24: package org.apache.bcel.classfile does not exist import org.apache.bcel.classfile.Field;

ava:2054: warning - Tag @see:illegal character: "63" in "https://issues.apache.org/bugzilla/show_bug.cgi?id=45227" /Users/depasqua/Desktop/comtor/apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitTask.java:2054: warning - Tag @see:illegal character: "61" in "https://issues.apache.org/bugzilla/show_bug.cgi?id=45227" /Users/depasqua/Desktop/comtor/apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/ant/types/Assertions.java:157: warning - @return tag has no arguments. /Users/depasqua/Desktop/comtor/apacheAnalysis/files/ant-1.8.2/apache-ant-1.8.2/src/main/org/apache/tools/ant/types/selectors/modifiedselector/ModifiedSelector.java:495: warning - @return tag has no arguments. 152 warnings