Thursday, August 18, 2016

It's almost over

Finally after 4 months GSoC the end is near. It was a awesome experience to work with SBML and JSBML team and how much work and efforts they put in this project and what is to work in a collaboration. If it is possible I would like to continue my work after GSoC.

Repository Information

Background

Because there have been of a lot of changes since the first post, I would like to introduce Deviser's code generator to produce Java code once again.

SBML Packages

Computation modeling has become a crucial aspect of biological research, and SBML (the Systems Biology Markup Language) has become the de facto standard open format for exchanging models between software tools in systems biology.

Since SBML Level 3 is being developed as a modular format with optional SBML Level 3 packages are available to extend the focus of the core SBML representation. Deviser is a new code generation system developed to facilitate the development of SBML L3 packages by helping to automate the generation of specifications, UML diagrams and library code.

The current fork extends Deviser's code generator to produce Java code that can be integrated into JSBML, the pure Java API library for SBML. This allows Deviser to target both libSBML and JSBML. At the current stage it generates the foundation of the packages, which facilitates their implementation.

Using Deviser for Java Code Generation

There is command line version of the function that can be used to invoke the deviser functionality for generating Java/JSBML code from the XML file. This is the deviser.py file found in the generator directory.

deviser.py [--generatejsbml][--latex] input-file.xml

Requirements for the fork

Deviser requires javap for the java code generation at the current phase. Javap is available with the JDK and the following is required for Devisers' java code functionality.

Also set up JAVA_HOME environment variable for your operating system, if you are working under Windows, here is a good tutorial on how to set up. And for Linux: Ubuntu/Debian.

Current State

At the current state Deviser generates compilable templates, which serve as a foundation and facilitate the implementation of the following packages :

Qualitative Models package (qual, for short) allows species in a model to have non-quantitative or non-continuous levels. (Chaouiya et al., 2013)

Groups (groups, Hucka and Smith, 2013) agglomerates SBML model elements and can be linked to annotations and SBO terms (Courtot et al., 2011) to contextualize sets of objects for other programmers and modelers.

What is left to be implemented

Specific parts of the packages are left to be implemented.

Flux Balance Constraints (fbc, Olivier and Bergmann, 2013) encodes components for constraints based modeling (Lewis et al., 2012), which employs a class of models in which the canonical stoichiometric relations between reactions and metabolites are specified as constraints for mathematical optimization.

I would also like to optimize 'imports' parts, as well as to find an alternative to javap, which is used extensively, so deviser does not have any dependency for Java/JSBML code generation.

Java Tests

For running Java code generation tests, please head to generator/tests/test_java_code and run

run_java_tests.py

If there are any errors, it will also show the code differences.

Final Notes

So,this is the end. As I said, it was a great summer and I had a lot of fun working on this project. I could recommend everyone who wants to improve the coding skills to apply for the Google Summer of Code. There's nothing to lose, but you will gain a lot, specially invaluable knowledge and experience. And even if you are not a system biologist, it is a promising field, so checkout the projects of the SBML and JSBML team.

Acknowledgments

I am very grateful to my mentors Dr. Bergmann, Dr. Keating, Dr. Dräger and Dr. Rodriguez for the invaluable help and patience.

Tuesday, August 16, 2016

Important bugs were fixed in readAttributes and toString method generators.

The constructor generator was updated as well as the hashCode generator.

Tests were set up and run fine both on Windows and Linux.

A prototype README.md version was created.

Updated some parts of the code comments, but still there is work to be done.

The good news is that the parser generator was finalized and that generated code by deviser for qual, dyn,distrib and groups packages are compilable.

The plan is to spend the next 2-3 days cleaning up the code and updating the comments, since deviser is quite complicated and there are tricky moments. They are available at WIP branch of the fork repository.

Wednesday, August 3, 2016

One addition to the tests was creating a new function for comparing the generated files with the test units, the new function uses difflib from python and now it is possible to see the differences and the lines if there are errors.
And the good news are that the codes generated for qual, fbc, dyn, distrib and groups packages are compilable(53dd6b7).

So far from the packages only 'spatial' is left to be adapted, that means the necessary functions need to be adapted for the code generation. And there is also the question for the generation of the parser.

Tuesday, July 19, 2016

Still Work in Progress....

Tests

Last week a compilable class goal was achieved.So this week was about setting up tests.

Since deviser generates and select random UUID and prime number, the two methods were adapted for running tests. In the case of prime number, random.seed was set to 0 as for the UUID to a fixed number,as random seed would not affect it.

The foundation of java tests served test_cpp_code files, which were adapted for the java files generation. The path separator was changed to os.sep, so it uses OS specific separator.

javap parser

jsbml jar was put inside deviser package and javap works under Windows as well.

Tuesday, July 12, 2016

Compilable Class

Deviser generates a compilable QualitativeSpecie.java(Fig.1). The class also worked as a replacement in the official jsbml code, but since deviser does not generate not generate QualitativeSpecies(species) constructor, that part of the code had to be commented in the official jsbml code, after that the compilation was successful(https://gist.github.com/hovo1990/c3ce45eb0d61cefe5c2440d65a45d5be).

Fig.1 Compilable QualitativeSpecies.java

UUID generator

Dr. Bergmann gave valuable help and recommendation for the generation of UUIDs.

Python includes uuid module, that can be used for the generation of uuids, but since Java uses long 64 bit, while python generates 128 bit, the generated uuid had to be masked.

After changing to 56 bit, java compiler would not give more errors for being too long.

hashCode method changes

Generated prime numbers were changed to a global variable, so that the numbers are only generated once.

Implemented Methods

Here's the list of the methods, that were implemented during this week:

Tuesday, July 5, 2016

Organize helper functions

Removed duplicate codes and placed frequent helper functions inside a new module: jsbmlHelperFunctions.py, which includes functions for determining whether a method has to be overriden, javap argument parser, get javadoc comments, finding methods with same name that take different argument and prime number generator.

Constructor generator

Deviser now generates copy constructors.

Method generators

hashCode method

The generator uses algorithm found from stackoverflow. Right now generates up to 10 to the 6th power(has to be changed) and than uses pythons random.choice method for selecting a prime number, but this part needs to be discussed with the mentors.

Things to do

With the generation of the new methods, previous errors during compilation would vanish, but new ones appeared.Only 3 methods are left to be implemented: readAttribute, writeAttributes and toString.

Finish readAttribute method, which turned out to be a little problematic, write writeAttributes and create the last method generator,which is for toString method.

As for fixing the order of the functions, the idea is to create a dictionary with the function name as key,sort keys and write methods, but that approach would consume more memory, I think it'll be nice to discuss it during TC.

Tuesday, June 28, 2016

Class methods detector

This week involved devising a system for automatic override key detection.

For that I came with the idea to create javap parser and look inside import modules and see which methods are abstract.It works, but maybe I overcomplicated things slightly. For the future I think I will insert all the data from javap for a particular class inside a dictionary. It will change the implementation, but the logic about how to detect override methods won't.

It can also be used for the detection of the missing constructors,that need to be generated as well as get Instance methods.

Mandatory methods generator

This weeks also included the creation of the Mandatory methods generator.

Dr. Bergmann gave a valuable insight about devisers attribute['reqd'] key,which facilitated the detection of return value for the method.Here's the link.

Things to do

Go back to constructor generator.

In the case of QualitativeSpecies.java, the current generator creates setCompartment(java.lang.String) method, but during compilation I get an error(QualitativeSpecies.java:36: error: QualitativeSpecies is not abstract and does not override abstract method setCompartment(Compartment) in CompartmentalizedSBase), which is visible during javap output,so that is something that has to be addressed. Also will take care of the generation of Instance methods.

Monday, June 20, 2016

Unfortunately last week not much was done because I had a exam at the university.I passed it and got an idea.

The idea came to fruition when last week Dr. Dräger mentioned about java reflection and jython, but I thought there should to be an alternative.

As to me the alternative can be pyjnius, which is a Python module to access Java classes as Python classes using JNI. It has an active community that comes from Android development, easy to use and does not depend on many dependencies.
These are the dependencies.

Java JDK(OpenJDK also works)

Cython and six

Setup JAVA_HOME environment

Setup JRE_HOME environment

Just with a couple of lines it's possible to see all the methods of the class(raw),
but again it turns out that java reflection is not enough for imports.For example, IDEs' such as Eclipse have their own parser to determine what to import. Still this approach can be used to get instance() methods which are not available in the Deviser xml descriptions.

Tuesday, June 14, 2016

This week involved a lot of investigation and figuring out how to use Devisers' key attributes to import modules necessary for JSBML.
Here's the summary what was done during this weekend.

Removed mId and mName from variable generation

Removed Id and Name methods generation from SetGetFunctions.py

Added simple constructor for JSBML

My previous idea to create the jsbml class hierarchy was not sufficient to determine the import modules unfortunately. So I examined again if there's any pattern for Deviser 'type'( here is the Speadsheet), but there is none, that could be used.

Looks like the best approach will be hybrid and one idea is to use attributes' 'name' or 'xml_name', for example if it is 'compartment', then import org.sbml.jsbml.Compartment and org.sbml.jsbml.CompartmentalizedSBase or if it is 'reaction' then import org.sbml.jsbml.Reaction and associated modules.
The idea is to create a dictionary with the package name and attribute name and to use previous tree structure to detect if it is an interface or not, but this part needs to be discussed with the mentors.

Tuesday, June 7, 2016

Opening and closing brackets were modified,so now it looks like Java code.

Code for writing import statements and class header was written.

In SetGetFunctions.py, unset was fixed, now throws "firePropertyChange".

But the biggest challenge was how is deviser supposed to decide which modules to import based on the attributes.

At first, the idea was to create a method similar to expand_attributes and expands_class, but soon found out it wouldn't suffice and there would a lot of problems on how to import.

Fig.1 JSBML Full Type Hierarchy

Thus I decided to use a tree structure to store the Full type Hierarchy of JSBML. Next step was to decide, whether to create my own tree structure or to use template. In the end I decided to use a template, but how without importing modules that are not available in Python and I found a gist summary that uses pythons' collections module, which includes High-performance container datatypes.

As for the keys, decided to use this ones:
['name'] = name of the Class or Interface
['hasParent'] = True or False
['parentNode'] = key name of the parent node
['hasChildren'] =True or False
['childrenNodes'] =List or None
['isInterface'] =True or False
['parentInterfaces'] = List or None
['childrenInterfaces'] = List or None
['isUniqueJSBML'] =True or False (if True, that means only available in JSBML)
['level'] =integer (to describle on what level)
['libSBML_analogue'] if there's analogue in libSBML

Here's the full implementation of the tree.
Next step is to figure out how to import only necessary modules.

But there were way more changes and other changes have to be done as well.

Refactoring of the variables and class names has been performed.

In BaseJavaFile.py: It was modified, but the main changes were done to the expand_attributes method, also a new dictionary value was added the "JClassType" for attributes. Since Java has both the primitive and the Class. Example int and Integer.

Major changes in the SetGetFunctions.py, that included modifying write_set,write_get, write_is_set, write_unset and their submodules.

Monday, May 16, 2016

It involved reading and comparing between the Deviser generated packages with the JSBML examples as well as delving in the references of C++, Java, JSBML.The work was done on the 5 packages(fbc, dyn, distrib,spatial,qual).
The diagrams for the JSBML examples were obtained through ObjectAid in Eclipse.

High resolution PDF links are posted.
Here's the meaning behind the coloured circles:

Green - similar in both LibSBML and JSBML
Blue - similar, but in a different location
Orange - similar, but with some differences
Red - not available in LibSBML

After that to have a better understanding of Deviser, which is written in Python, having a class diagram would be helpful.In the end I installed pylint, which includes pyreverse and here's a PDF diagram,but need to have thorough look at the options.Deviser classes

On a final note, the fork of the Deviser generates the JSBML package folder structure as described in the JSBML Manual.

Monday, May 9, 2016

There were couple of problems, that had to be solved.
Debugging with Spyder was not as effective, nor comfortable for big project like this, so I set up PyCharm Community Edition, which has very robust debugging tools.
Creating UML diagrams for the SBML and JSBML extensions by hand was not effective and because of that I decided to find programs for the automatic generation of class diagrams and looked through programs such as Umbrello, ArgoUml, SmartUML and others, but in the end I installed ObjectAid for Eclipse, and Oovaide(https://github.com/animatedb/oovaide)
Here's the summary what was done during this weekend.

Migrated from Spyder to PyCharm, Spyders' debugging tools are not as powerful as PyCharm.

Eclipse was set up and all JSBML tests were passed.(Fig.1)

ObjectAid for Eclipse was set up for the automatic creation of java class diagrams to aid analysis and comparison between SBML and JSBML extensions. (Fig.2)

Oovaide was set up for the automatic creation of c++ class diagrams, but it's not as powerful as ObjectAid.

Sunday, May 1, 2016

GSoC has been going on for a week, and the SBML and JSBML team and I have
been getting to know one another through dozens of emails. Since this was my first time working with a highly
established project such as Deviser and JSBML, there was a definite learning curve. Dr. Bergmann, Dr. Keating, Dr. Dräger and the rest of the team were incredibly patient with me.

So here's the summary what was done last week:

Forked Deviser

Added placeholder code for the generation of JSBML code.

Created a UML diagramm of the Deviser's SBML code generation to see the interaction between the modules.

Tuesday, April 26, 2016

My name is Hovakim Grabski. I am Ph.D. student at the department of Medical Biochemistry and Biotechnology at Russian-Armenian (Slavonic) University.I hold a specialist's degree in Bioengineering and Bioinformatics from the Russian-Armenian (Slavonic) University. This summer, I am being mentored by Dr.Frank T, Bergmann and Dr. Andreas Dräger, both part of the NRNB. My project focuses on Deviser,which is a new code generation system being developed to facilitate the development of SBML L3 packages by helping to automate the generation of specifications, UML diagrams and library code as well as to provide a fast, efficient means of changing both API code and specification documentation to facilitate the update of documentation and implementation necessary as the package itself evolves. The code generator currently targets ​ libSBML only and the goal is to extend Deviser's code generator to produce Java code that can be integrated into JSBML​, the pure Java API library for SBML. This will allow Deviser to target both libSBML and JSBML​ (​https://github.com/sbmlteam/deviser​)

I am very enthusiastic about the opportunity to work with Dr. Bergmann and Dr. Dräger and the rest of the NRNB and SBML team. I am sure that it will be a great summer.