Moodle plugins directory: javaunittest

Hallo Moodle Users. We have developed a new question-type called javaunittest and want to share it with the community. The plugin is a new version of the sojunit questtion type developed by Süreç Özcan for Moodle 1.8 (https://moodle.org/mod/forum/discuss.php?d=102690).

For computer science courses this question type allows a teacher to create Java questions and test the knowledge of students about Java programming. The students type source code for a given interface in Java and the response gets graded automatically. This way the teacher can save time for corrections, which is particularly important when large number of students visiting the course.

Figure 1. illustrates the editing view of the javaunittest question type. In this example the teacher created two javaunittest questions. In the first, the students have to implement the factorial function, in the second, the reverse string function. Every question has its own answer field with an interface given by the teacher.

The student implements the javaunitttest question on the question editing form and writes the response in the answer-field. Figure 2. illustrates an attempt.

After finishing the attempt the student submits the response. In case the student response compiles the response gets graded automatically with a JUnit test implemented by the teacher. The JUnit test evaluates the response and returns a value between 0 and 100%, which represents the correctness of the student's Java code. After that, the student's grade is computed automatically by Moodle from this percentage value. With compilation error the student's answer is graded as ,,wrong” with 0% correctness. After executing the grading, the student can view his given answer with a feedback about the test. The student gets all the questions with question text, the student-response and the compiler- or JUnti execution-output. This is illustrated in figures 3 and 4.

You need the following to use this question type:

Java and JUnit have to be installed on your server. In the configuration file the javac, java and junit variables have to be initialized with the proper value. Fore more information see the README.txt file in the plugin's directory.

In order to get an automatic grading for this question you need to create and implement a JUnit test-file. This file needs to match exactly the class and function names used in the source code given by the teacher. This code serves as interface between the JUnit test and the student's response. In the EXAMPLE_FILES directory, three examples were defined. For more information see the README.txt file in the EXAMPLE_FILES directory.

Awards

Comments

Thanks, i downloaded and used git to find out the remote. I've added it to your plugin entry here now. It's generally preferred not to include '.git' directory in zips, you may want to address that by adding a line with the git url to README.txt. I shan't delay this any longer.

This plugin contains potential privacy issues:
How could a malicious user exploit this plugin? Could a user submit code that could cause a DOS attack with a loop, have direct access to files on the server, exhaust memory resources. It seems that it would be better if the evaluation and execution of the code made use of an external service. I am interested in comments from those more familiar with security.

In the plugin, there is a polifile. It is a security manager, which checks for non-proper java code and prevent the execution of program code, which could damage the server. Furthermore, since the plugin creates and works in its own directory, in which the files are created on the server with user role, the plugin is not able to execute any program code, which could damage the server, since the test is executed with user role.

Has anyone found a way to set run time limits on each test case in the TestFile? I have used @Test(timeout = 200) and it seems to not respond to this. The plugin is working perfectly, but I need it to only run each case for a limited amount of time. Thanks.

@stacey mr: soon we will release a new version which will allow to sent the code to another server (via curl) to another server. this will also allows to terminate the entire test. Apart from this Timeouts for the Test will be provided.

@cse batch: C programming tests will not be supported in the current and upcoming release. However the execution on a different server might be extended to setup a virtual machine to run the c code. This seems necessary to us as c does not provide a security manager

Hi Claudia,
Testing multiple outer classes is currently not possible. Our current recommendation to circumvent this is is the use of inner classes. We are currently working on a version which wraps the code in the window in some additional hidden instructor-given code.
Best,

Hello, and thank you for this amazing plug-in. I have used it successfully in Moodle 2.8. I have a feature request: it would be really great if you could add an feedback option for the results of the unit tests. For example:

sum(2,3); expected: 5; actual: 0;

I think this would help students diagnose their errors on questions where more than one submission is allowed.

Currently there is no feedback for the message string parameter of the overloaded assert*()-methods. We like the idea of a static feedback language to avoid any attack vectors against the escape functions or to trick the regex used for parsing the junit output (to generate grades and feedback). This also avoids to get any system information which may be interesting for an attacker.

For the next update we have plans to allow string literals as a first step (like assertArrayEquals("Array is ruined", ...), but not AssertArrayEquals(errstr, ...)) by preparsing and use of unique placeholders. And maybe the option to return the real junit assert error string (via regex) and/or stacktrace but with a global on/off switch for the admin and a warning.

Before reading what you wrote above, I created a new option to display the result string. It is very rudimentary, but it works with the existing code with minor modifications. I've only tested it with assertEquals, but I think it should work with other assert* methods. It basically looks for the existing strings in the junit error output like "expected x but got y" and outputs that text along with the String message in the assert* method.

I put a pull request on github for this. Take a look when you get a chance. I appreciate your feedback on it.

Hello Michael,
we've just updated our Moodle installation to Version 3.1. Is your Plugin available for Version 3.1?

Currently we've got an error by using the Plugin: JUNIT Test File Error: The submitted answer ist not compatible with the Test class or Test class can not be compiled (translated form german to english, I hope it is understandable). In our old Moodle Version (2.9) this question Type workes fine !

I am facing some problem while setting up coding quiz using JUNIT in Moodle.
When I am trying to compile the code then it give below error :
"'ulimit' is not recognized as an internal or external command,
operable program or batch file."