Subscribe To Our Blog

"Under the Hood" - Considerations While Selecting an Assessment Platform for Coders

Written By
mettl

If one were to ask what is the most important ingredient in making a technology product or executing a technology project, the prompt answer would be great programmers.

That said, good programmers are not easy to come by; any clues as to how would you plan to hire all those talented geeks out there when they are not even interested in your job postings on various job portals. Here’s a hint: programmers love challenges; be it in form of hackathons, codeathons or an assessment that truly measures their coding prowess; throw them a good coding challenge and there you have it - a landslide of coders ready to hack a code! Obviously for you, a great pool of skilled coders ready to be hired.

To simplify your “geek-selection” problem, there are several software development/coder assessment platforms out there doing the rounds on the Internet. But, spotting the right online assessment platform is not as easy as it sounds. As recruiters and technology managers, you need a coding assessment platform that simulates real-world coding environment - the way a software developer will code at work. Code simulator platforms available in the market today fail to create an environment that corresponds to certain or all aspects of a real life coding software. If you take example of pilot training, simulators are used to train the trainee pilot, but the plane simulation is built in such a manner that it emulates the cockpit, air controls etc., as it would be available on a real plane. The idea is to bring training as close to reality as possible, so when a trainee/intern, fresher or experienced hire is hired he already knows what platform and tools he has to use and how.

Our product and engineering folks have combined a few business and technical considerations to be kept in mind while selecting your platform for running coding assessments for recruitment or training / learning effectiveness; we have seen that these considerations are typically missed in the hype and would want you to pay due attention to all these subtle points;

Coding Question Support Depth

1) Can you create your own questions based on your own requirements

Should the question creator be very technical to create questions.

Are you free to you use your own data structures while creating questions; for e.g., if only STDIN is available, the question creator is forced to use data structures that are easily converted from a String

2) Can the platform support testing of front-end coding skills

Is there scope for languages like HTML5, CSS3, JavaScript etc.?

3) Can real-life problems be created that involve code-snippets integrated with a database

Is Read/Write from/to a database supported in the platform using industry standard ways

4) Does the interface have support for multiple classes and files

5) Is integration with standard IDEs like Eclipse and Visual Studio supported

This helps the candidate quickly adapt to a new problem solving environment

6) Does the offering have “project” support

Can custom source code be attached with a question for reference / use by a candidate

Does the system allow multiple classes to be included along with the template from which a candidate starts attempting the problem

7) How many data types are supported as input and output arguments when generating a code snippet for the required function

Are common data types like long, float, char, byte and their equivalent wrapper classes available;

How many array types are supported

Are common collections like Map, Set, List available

8) Is it possible to attach additional resources like a text / xml file with the problem

This helps the problem creator to have more leeway in designing problems for the candidates

9) Does the candidate need to write unnecessary boiler plate code to read inputs when a template hasn’t been specified

For example, given a map of employee id and their names, how easy is it to sort them on the basis of employee names

10) Is the candidate handicapped or are tools available for a developer under normal circumstances provided to the candidate while attempting a coding assessment

Is Auto completion of code context sensitive and does it work in all scenarios

11) How many value added services are provided through the platform for grading

Is feedback of the candidate coding style and static code analysis available

Are customized grading options available

12) Can the candidate while attempting a question check against any arbitrary input to ensure that the generated code passes checks that she wants it to pass

13) Are standard unit testing frameworks available

For eg, jUnit in Java

14) Is support for common frameworks of a programming language available

For e.g., is it possible for a candidate to write code involving WCF in C# or Spring enabled solution with Java

How many languages are supported by the platform

15) Does the platform support reusability

Are coding questions re-usable; ideally they should be outside the scope of an assessment and should be possible to combine a question in different assessments

What is the depth of sample content available for a test author to experiment

16) Hygiene elements

Support for CTRL+Z to delete the last typed word

Line numbers starting from a logical number instead of an arbitrary number because of the hidden template

Availability of common shortcuts used in popular IDE like Eclipse or Visual Studio

Creating Assessments from Questions

1) Does the platform have the capability to logically group multiple questions in a section (which may be timed)

In addition to coding questions, can the platform help the test author combine other question types like Multiple choice questions (MCQ) and other skills like psychometric skills to get a full picture

Does the platform support rich questions with media elements like audio / video

2) Does the platform account for all scenarios of test completion

In addition to normal submission or time expiry, are events like Window closure detected

3) Proctoring support - how easy is it to prevent cheating and plagiarism even when the test is being taken remotely

Are features like automatic face matching, multiple face detection and browser locking available

Does the platform allow a test administrator to remotely finish a particular candidate’s test

What are the other tools available to the test administrator when an unfair advantage for a particular candidate is detected - live chat, live video capture, live screen capture

4) Can a test author collect customized information from a candidate before the start of the test in specialized text, yes/no and date input fields.

5) What happens in case of network disruption

If the candidate is unaware, the test continues to proceed and all the time spent is eventually lost

6) Access control

How strong is access control and what are the rules that you can impose on other team members using the same account

Can you restrict access based on various features, modules and data to other people in the organisation

7) Are the assessments editable entirely at any stage of their life cycle

This might be double edged, as on one side you get flexibility and on the other side there is potential that reports across candidates might not be totally accurate

8) API

How extensive are the APIs; can it pass data to a remote system when the test concludes

Can it be integrated in a low cost manner to an existing system

These are the broad technical and business specifications that you as a hiring manager, training manager or a senior developer should keep in mind before investing in a technology that is aimed at hiring the better of the “technical world”.

Hope that you find it useful. Think we have missed something important? Comment and let us know, we will try to help you. If you are looking for a solution on the similar lines do checkout Mettl’sonline coding assessment platform.