things I learned throughout the week

Menu

I had downloaded a open source tool pylint. When I got it to run, I was always given this stack trace “Too many open files”. After searching for a clue on the web, I was pretty positive too many file descriptors created by the tool since I saw a similar problem. Wanting to be sure I did:

$ watch lsof +D .

while running the tool. It looked right the output filled up the screen.

With the help of my coworkers, I discovered that there is a setting to increase the max opened files allowed bye a process. The command is

It’s a bash command to limit system-wide resources. Another thing that is new to me is that there is a concept of soft limit and hard limit. Not sure why there are such things, I seem to notice that the soft limit is what preventing pylint to work on huge number of files. It seems to me that the soft limit is the real max value that a user can set.

Below are ways to find out the lower and upper extremes.

$ ulimit -a -H

$ ulimit -a -S

Let say the soft limit for open files is 1024, you cannot do $ ulimit -n 1025 because that is greater than the soft limit. That was a problem for me since I needed more so what I need to do first is to increase the soft limit and then use ulimit to increase it for my session.

First of all I was amazed by the fact that FF has this hidden slides feature. I wondered what tool was used by the author to make the slides. Does anyone know?

Anyway, it is an interesting set of slides on unit testing. And in there, there are some slides on Mock objects. I haven’t fully understood what they are yet, but I think Mocks are objects in which expectations can be set to their interfaces. The expectations are based on known specifications. The reason is that so that tests can be written for the specifications before they are implemented. Right now, I am still not sure the benefit of doing this yet.?.?. This is probably not new to many people but this is the first time I heard about it. It sounds like a useful concept. If anyone can point me to good readings/articles/books on mock objects, I will definitely appreciate it.

On one of my previous posts, one of the remaining items to do is to have server generated web pages to fetch reports from the data storage and to bring it to the user end. For this implementation, the project will be using brasstacks, couchquery, and webenv. So far web pages for simple views have successfully been implemented using brasstacks. As a result, the focus now is on the scenarios how the information will be used.

Currently focusing on:

given a build, finding its last previous build.

given two builds, finding the difference at individual test level as mentioned in my last post.

To find the previous build, my idea, which I don’t know if it will be feasible or not, is to have a view where the keys will be the necessary metadata and the value to be the build id. When querying the view, filter the rows with the metadata of the current build. The metadata that identifies a particular build will be the product, the OS, and the test type. The question is whether all these can query arguments for key= like (key=[product, os, testtype]). So far from what I have seen I can set the key to be doc.build such as emit(doc.build, doc) and set the key to be a specific build by specifiying key=”sample_build_id”.

To get the previous then, one approach is to iterate the rows and find the timestamp that matches with the timestamp of the current build and return the row after that which would then be its previous build. We’ll see how it goes once couch.io is back up.

On my last post, I talked about the Comparing Test Results project that I am currently working on. In that post, it also mentioned parsing of logs to extract useful information. So far this has already been implemented and tested with data from Fennec builds. The intention of this post is to explain how the parsing is done so it can be a reference for improvements that may need to be made to extend this project to other builds.

There are two parts to the parsing. The first part is to look at the entire log and to find metadata used to characterize and identify a set of data. Here the code is looking for: the build ID, the product, the operating system, and the type of tests that were run.

To get the build ID, the code is looking for the line that contains the string “BuildID=” such as this “BuildID=20090715053040”. The information to be extracted will then be whatever after the character ‘=’.

Similarly for the product, the code is looking for the string ”Name=”.

For the OS and test type, the code is looking for the line that contains the string “tinderbox: build: ” such as this “tinderbox: build: Maemo mozilla-central crashtest”. The OS will be taken from the first word after the string “tinderbox: build:” and the test type will be taken from the last word.

The second part is to parse the log per line to find lines that contain the output of a test run. On each line, if it’s an output of a test run, it is looking for: the result of the test (PASS/FAIL/TODO), the location of the test code (normally the path to the test source file), and if any the message that may describe the intent of the test.

To determine if a line is an output of a test the code is looking for either of these strings: “TEST-PASS”, “TEST-FAIL”, “TEST-UNEXPECTED-FAIL”, “TEST-TIMEOUT”, “TEST-KNOWN-FAIL”.

The line is then split into three sections as separated by the divider ‘|’. The first section will determine the code will go into either of these conditions: the test passed, the test is marked as TODO, or the test failed.

The second section is simply taken as the location of the test code. Since only the relative path is needed, the beginning is striped as it appears for ‘reftest’ and ‘xpcshell’.

The third section is taken as the message, if any, from the test. It is taken if the test did not pass.

The code also increments the count of the number of passes, failures, and todos.

As a result, in any of the three conditions, the code will see if it is a new test. If it is, it is added into the data structure with a count of 1 and if it is not a test that passes, its message is added as well. Otherwise, it will update the count of an existing test and if it is not a test that passes, the message is appended and separated by a coma.

Everything mentioned is not final, still continuously being improved, and is open to suggestions and extensions. The post will be updated as the code is updated.