John Lueck (F'11)

Objectives and Key Results (OKRs) for the Fall 2011 Semester
Objective 1: Getting Started
 * Key Result 1: Become more familiar with Javadoc (began on 9/11/2011)
 * Key Result 2: Get better understanding of COMTOR code structure/conventions (began on 9/12/2011)
 * Key Result 3: Learn the basics of using Git (began on 9/15/2011)
 * Key Result 4: Create diagrams and pseudocode for controlflow module (began on 9/12/2011)
 * Key Result 5: Research how java compiler traverses source code, specifically process it uses to determine which method to use (began on 9/12/2011)
 * Key Result 6: Learn how to use ANTLR-based parser

Objective 2: Implementing Controlflow Module
 * Key Result 1: Create javadoc tag for controlflow (started and completed 10/20/2011. Simpler than anticipated)
 * Key Result 2: Implement controlflow analyze method (began 9/16/2011)
 * Key Result 3: Make method for controlflow to compare its result with the user-provided tag
 * Key Result 4: Implement other methods as needed (most likely will include: setGradingBreakdown, getGrade, setGradingParameter, and setConfigProperties. May change based on COMTOR parser changes with ANTLR)

Objective 3: Adding Complexity to Controlflow Module
 * Key Result 1: Make controlflow able to show if there are conditions or repetition in the sub-methods that it notes.(e.g. @controlflow X.bar!Y.foo!Z.so would be a standard notation for a method and methods it calls, but what if controlflow could show in comment if some methods were called multiple times or based on a conditional operator? e.g. @controlflow X.bar&Y.foo::Z.so could note that Y.foo is called conditionally and that Z.so may be called more than once
 * Key Result 2: Allow controlflow to check the methods the tagged method calls for accuracy as well. (e.g. lets user know if Y.foo in example above is not noted correctly, which means user's notation of X.bar is still fundamentally wrong.

Progress Blog
11/10/2011 - Variety of test cases made and run. There appears to be no exploits in method call naming that cause ControlFlow to give an incorrect return. (in relation to the current scope of checking tags without looking at access modifiers or the source code.)

11/9/2011 - ControlFlow module is now theoretically able to process all method calls of the following types:

package.Class.field.method package.Class.method Class.field.method Class.method field.method method

Where "method" means a method or constructor and "field" can be either a single field or multiple fields being called by one another. Will develop test cases to test robustness of each method call type.

11/2/2011 - tags of type package1.Class1.method1!package1.Class2.method2... are being processed correctly. Need to slightly change way that tags are processed so calls of type Class1.method1!Class2.method2 read properly. Since ClassDocs constructed with Strings need to be fully qualified names, Class1 and Class2 as in the tag will be assumed to be a part of Class1.method1's package, and the Class will be created using the current package name appended to the class name.

11/1/2011 - fixed previous error, however, unable to find way to use javadoc to get the class of a field, which is fairly important for calls of type C.f.f.m, or any controlFlow using a field.

e.g. A, B, and C are classes, A has a field of class B called bravo, and B has a field of class C called charlie. A has a method a, with a control flow tag of a!bravo.charlie.c. Since there is no recognizable class name in the tag, the class is (correctly) assumed to be A. To check each field, my analyze method checks class A's fields for a field bravo, and bravo is found. In order to find charlie, however, I need a way to get the ClassDoc of B to verify field charlie, from javadoc. There does not appear to be any suitable method for this, as the only methods that seem to return either a ClassDoc or String with the proper name are containingClass and qualifiedName, and calling these methods referring to bravo only returns A and A.bravo respectively.

10/27/2011- code compiles and runs for tags of type m!m!m... but runs into errors when fields are introduced

10/26/2011- began writing code to allow control flow module to discern between packages, fields, and classes in a method call. Code compiles, but is untested. Will write and run test cases tomorrow.

10/21/2011- beginning to move code from early version made 9/16/2011 to the amazon server.

10/20/2011- first basic implementation changed; ControlFlow now finds its tags and prints them in Properties.

10/12/2011- ssh machine now working properly, wrote "Hello World"-style implementation of ControlFlow, mostly consisting of a stripped-down version of CheckForTags module

10/6/2011- Issues found in ssh machine. Were resolved up to the point of having comtor.jar able to build. Problem is now in classpath variable not finding javadoc library

9/29/2011- Approach towards working stand-alone changed to ssh connection into a linux machine with ant and comtor already runnning

9/26/2011 - 9/28/2011- Trying to get ant/comtor to install and run on either my windows or linux installation unsuccessful

9/23/2011- Attempts begun at making stand-alone version of comtor via Ant.

9/22/2011- Beginning to make code able to traverse javadoc. Need to find a way to process classes into a javadoc (RootDocs, PackageDocs, ClassDocs, etc.)

9/16/2011- Implemented code capable of parsing a tag string to find methods and classes within the tag.

9/15/2011- Began reading up on Git

9/12/2011- Found books in library about writing compilers, considering syntactic analysis through parsing to determine when a method call has been made. Will require use of other techniques to determine what datatypes parameters are to find out which overloaded function is being used.

9/11/2011- Started learning about javadoc taglets and doclets using resources at oracle.com Looking for resources that will explain the process java uses to determine which method to run in cases of overloading, calling a superclass, etc.

ControlFlow Module: Method Lists
Note: the word "constructor" is almost never used in any parts of the description below, however, ControlFlow is currently just as capable of checking constructor calls for validity as it is methods. So, in the descriptions of control flow tag checking below, the word "method" means "method or constructor" in most cases.

Summary
The goal of the ControlFlow module is to evaluate a programmer's documentation of a given method's control flow. So far, documentation of control flow is very simple and follows the form methodCall1!methodCall2!methodCall3... in which methodCall1 would be a call to a method that uses the method referred to in methodCall2, methodCall2 then using the method in methodCall3, etc. Tags can have any number of method calls in them. So far, ControlFlow is able to determine if a tag is valid based on whether all the method calls listed within it are valid. The major shortcoming of ControlFlow at the moment is that it is not capable of parsing source code on its own to determine validity, so while it is able to state whether a tag uses methods that exist or not, the module is not able to truly determine if a tag is actually correct, while it is able to spot come errors in the user-provided tag.

Methods used and to be used by ControlFlow
These are the methods ControlFlow uses, each with a short summary of what they do and how far along they are in development

Methods required of ControlFlow as an implementation of the ComtorDoclet interface
Analyzes the source code parsed by Javadoc as a RootDoc object, focusing on checking if any control flow tags within the RootDoc are correct or not as far as the current implementation can tell. The analyze method prints its findings to a Properties object, which is then returned. The ControlFlow implementation of analyze uses the processTag and processMethodCall methods heavily. Analyze's structure is basically that of a loop iterating thorugh each class, and each method of each class, looking for control flow tags. Within analyze, processTag is used to process the contents of each individual tag.
 * public Properties analyze(RootDoc rootDoc)

Currently an empty method. Upon completion, this method will set the maximum attainable grad for a given section, through the parameters passed to the method
 * public void setGradingBreakdown(String section, float maxGrade)

Now an empty method. Will return the grade for the doclet the user received based on the accuracy and frequency of @controlFlow tags when implemented.
 * public float getGrade

Not implemented yet. Sets the value of the grading parameter named in String param equal to String value.
 * public float setGradingParameter(String param, String value)

Also an empty method as of now. Loads configuration properties from a configuration file
 * public void setConfigProperties(Properties props)

Methods used by ControlFlow not included in the ComtorDoclet interface
This method takes apart the control flow tags, dividing it into the individual method calls the tag is comprised of. The method processTag starts at the beginning of tagContents, and analyzes each character until it reaches a token separating method calls (currently, the only one is '!') or the end of tagContents. Each time processTag reaches either of these things, it sends the substring from an int labeled tokenBegin (initialized at zero) up to the index of the String reached before reaching the end of tagContents or an exclamation point. The substring is sent to processMethodCall, and tokenBegin is then set equal to the index plus one. A boolean value tagValid is initialized as true in processTag. If any calls to processMethodCall return false, indicating an incorrect method call, tagValid is set to false. This repeats until the end of tagContents is reached. If any method call in a given tag is considered incorrect, processTag considers the entire tag to be incorrect and writes a message indicating this to the Properties object. Later in development, processTag will also most likely send a score for the tag back to analyze.
 * private void processTag(RootDoc rootDoc, int classID, int methodID, ClassDoc taggedMethodClass, String methodName, String tagContents)

The parameters of processTag function as follows:

-RootDoc rootDoc - same RootDoc object received by analyze. Not used by processTag at all, it is passed on to processMethodCall

-int classID, methodID - the classID and methodID of a given class or method are determined by which point in the analyze method they are iterated through. classID and methodID are independent values. They are used to label output in the Properties object that will be returned by analyze. These are also passed on to processMethodCall.

-ClassDoc taggedMethodClass- ClassDoc of the class of the tagged method. Not used by processTag, passed to processMethodCall.

-String methodName- also passed to processMethodCall. String representation of the name of the tagged method. -String tagContents- String representation of the text contained in the tag processTag is checking the validity of. If a javadoc comment were to contain the phrase "@controlFlow bar!foo", tagContents would equal "bar!foo".
 * private boolean processMethodCall(RootDoc rootDoc, int classID, int methodID, int partID, String methodCall, ClassDoc taggedMethodClass, String callingMethodName, boolean firstMethod)

This other major helper method in the ControlFlow module is used by processTag to check each section of a given control flow tag. The method processMethodCall is similar to processTag in some ways. The String methodCall sent to processMethodCall from processTag is the substring described earlier, which should consist of one of the method calls found in the tag. Each character is checked, but processMethodCall starts at the end of the String and stops at either a period or the beginning of the String. The character being checked is tracked through an index. When the first period is found, or processMethodCall reaches the beginning of the String without finding a period, processMethodCall gives the name of the method in the method call equal to the substring from the end of the String up to the value of the index (the location of the first period, if not, the end of the String). After finding the name of the method in the method call, processMethodCall tries to find a package and/or class name in the remainder of methodCall.
 * First, processMethodCall checks if the remaining portion of methodCall is the qualified name of a class existing in rootDoc. If it is, processMethodCall creates a ClassDoc from this qualified name, which it will check for the method in.
 * If not, processMethodCall checks if the package the tagged method was found in (the package of taggedMethodClass) exists. If it does, the name of the package is appended to the remainder of methodCall, and processMethodCall checks if there is a ClassDoc with this name. If there is such a ClassDoc, the ClassDoc generated by this query is now the class the method belongs to.
 * If none of the above methods work for finding a class, processMethod call will place the portion of methodCall between the last period found and either the next period or the beginning of the String, if no period is found, into an array titled fields. This process repeats until either a class is found, or until the end of the methodCall String is reached and the class is simply assumed to be the same as that of the tagged method.


 * After finding the name of the method and which class it belongs to, processMethodCall will check if the field, or sequence of fields, in the method call is valid. The field found last (the one furthest left) is first compared to every FieldDoc in the current class(found above). If the first field matches, the current class becomes the type of that field, and the next field (if any) is looked for in the class the previous field was a type of. This is repeated until either all stored fields have been found correct, or one field is determined to be incorrect.


 * If a field is not correct/does not exist, processMethodCall writes a message to prop indicating such and returns false.


 * If all fields are found true, it is checked if the method exists within the last current class (if there were fields, and the fields were all found to be correct, this will be the class the right most field is a member of). This is done by looping through the MethodDoc for the class being checked, until one with a name matching the one processMethodCall is looking for is found. A loop is also run for all the constructors of the same class, looking for a match.
 * If the method name in the tag matches the names of one of the methods or constructors in the class processMethodCall determined it belonged to, the method returns true and writes to the Properties object that that particular method in that portion of the control flow tag was found.
 * If the method name is not found, processMethodCall returns false, and writes to the Properties object that the method was not found.

parameters of processMethodCalls are as follows: - RootDoc rootDoc- Same RootDoc passed to analyze. Used to verify if a class with a given qualified name exists in the RootDoc, and if a given method exists in a class.

- int classID, methodID, partID- classID and methodID are sent to processMethodCall by processTag from analyze, while partID is created inprocessTag to keep track of which method call is being processed. All three are used in conjunction to determine which element to write to whenever the Properties object returned by analyze is modified.

- String methodCall- Created in processTag, this String represents the portion of a control flow tag between two exclamation points or an exclamation point and one of the ends of the tag. This is what processMethodCall is designed to validate.

- ClassDoc taggedMethodClass- The ClassDoc representing the class the method with the control flow tag is a method of. If no class can be found in methodCall, this class is assumed to be the class of the method in the method call.

-String callingMethodName, boolean firstMethod- These are used when the first method call of a tag needs to be checked. The first method in any given tag should be the method that is tagged. If the method call is the first one in a given tag to be checked, firstMethod=true. If firstMethod=true and the method name found by processMethodCall does not equal callingMethodName, processMethodCall returns false and writes to the Properties object that the first method call did not match the method that was tagged.