How do you feel about spelling errors in code? I wrote about them for SubMain on their blog.

You’re checking out some code. It’s been a long time since you looked at this project. Or maybe it’s the first time. You point your editor at an interesting file name and double-click to open it. Your coffee is warm, your mind is clear, and you’re ready to go.

And there it is, sticking out like white socks in sandals. A spelling error.

How do you handle it? Do you fix it? Do you ignore it? Discuss it with the author? Bring it up during lunch when he’s not around?

What Really Matters?

Speaking personally, this isn’t an easy question to answer. I write code for a living, and I understand what it means to be in a hurry and not have the bandwidth to worry about spelling and human grammar.

But I write words for a living, too. Spelling and grammar errors scream out to me, like a crooked painting in the Louvre.

On the other hand, if there were a Nobel for typos, I would be a contender. I usually fix them before I check in code or submit something to an editor. But not always.

How you react to spelling errors in code comes down to how you answer another, more fundamental question. What’s really important when it comes to evaluating source? Whether or not it works? Whether or not it can be understood? How correct it is?

That last question is the tip of the proverbial iceberg. What does “correct” mean? Most answers would include references to functionality, tests, maintainability, and best practices. Where does spelling fall on the correctness spectrum?

Does spelling matter?

Code Is for Coworkers

A few years ago, Derick Bailey wrote that code is for coworkers. He was referring to how we construct source code and, by comparing obfuscated code to the original source, illustrated that we write code for people. The computer doesn’t care how readable code is, but for humans, readability is critical.

The ability to look at your code and ask yourself “will anyone else be able to understand this?” is what separates a pro from a beginner—or worse, a hack. If another developer can’t understand your code, it’s not maintainable. It’s worthless.

But can misspellings render code unmaintainable?

The Case Against Spelling Errors

One problem with spelling errors is that you have to be aware of them to understand the code. If a “widget” is a “wdiget,” you have to know to search for the misspelling to find it. So, if the author has misspelled an important object or concept, the error is a frequent obstacle to understanding and fixing the code.

But misspellings are rarely that easy to navigate around. Most spelling errors are inconsistent. Widget is usually spelled correctly in 90% or more of the places where it appears. “Wdiget” only appears in one or two locations. This renders those uses invisible from searches. Even the most sophisticated refactoring tools can’t reconcile a spelling error with the correct spelling.

If spelling errors are not fixed immediately, situations like the one above will start to proliferate, rendering parts of your code unmaintainable.

The Case for Letting Spelling Errors Go

Of course, if a spelling error is making parts of your code opaque, you have more significant problems. Code that is so fragile that a handful of misspelled words impairs its maintenance must suffer from other design issues.

Why are you relying on grep or other search mechanisms to navigate through your codebase? Even if you inherited the source from another team or developer, text search is only one of many tools. Object and file names, external documentation, debuggers, and profilers are better ways to figure out how things work. A brute force file search is a last resort.

Spelling errors are a deficiency, and we should treat them as such. We should fix them and make an effort to avoid them, but there’s no reason to hold them out as a special class of problem.

But Should You Hold Spelling Errors Against Developers?

Ah yes, the original question. I thought you might let me off the hook on that one.

No, you shouldn’t hold spelling errors against developers.

What if the offender doesn’t speak native English? English spelling is atrociously inconsistent and is often an exercise in memorization. A developer’s number one focus should be on writing good code. Mistakes in a second language should be forgiven. If you’re willing to help a coworker learn how to write code, you should be willing to help with English too.

Or they were in a hurry. A spelling error is, at worst, a minor bug. Have you ever written buggy code while you were in a hurry? Me too. How about when you weren’t in a hurry? Yeah, me too again.

If you’re asking this question, it’s safe to assume that the offender is someone you work with. If you can “hold it against” them, it implies some real-life consequences. Maybe they work for you, and you can make things difficult for them. Maybe they’re a co-worker, and you’ll stop trusting them.

Lighten up.

So What Should I Do About Spelling Errors?

So here’s an idea: fix them.

What would you do if you found an off-by-one error? An uncaught exception? A memory leak?

Regardless of how you feel about how serious a spelling error is, your first reaction should be to fix it. You’re in the code for a reason, and unless the goal is some recreational reading, correct the error. The only reason to delay fixing the spelling would be that it risks a regression, such as renaming an object or method that’s dynamically linked.

If you can’t fix it on the spot, you’ve got two options. Open a bug for it, or if that might cause embarrassment for someone, go to them and work out how to address the problem.

The key concept here is this: it’s a mistake. Nothing more, nothing less. We all make them. The best way to deal with a small error is to fix it and move on. The best way to deal with a larger one is to make arrangements to have it fixed.

If you’re working in a culture where pointing out mistakes is a problem, it’s time to move on. Enough said.

Can’t We Avoid Spelling Errors in the First Place?

Most code editors and IDEs have some degree of spell check support. It’s not foolproof, and most source code is going to be full of false alarms. But you can turn it on and work on training it to learn your dialect.

And of course, there’s GhostDoc. GhostDoc can help you build excellent documentation for your code, and it catches spelling errors, too!

What’s the difference between microservices and SOA? The two design paradigms have a lot in common.

That makes sense when you consider that microservices are an offshoot of the SOA movement. But there are essential differences between the two systems. Let’s take a look at the two different approaches to architecture and highlight where they differ.

Before we dive in, it’s important to note that neither architecture has a universally accepted definition. So, you could spend as much time debating the details of what microservices or SOA are as you could arguing their differences.

What’s a Microservice Architecture?

Search for “what are microservices?” on the internet, and you’ll see many different definitions. So, let’s factor in the common attributes and look to an expert for some help with defining them.

A microservice architecture builds applications with loosely coupled services. Each service has a single responsibility. So, they focus on one task, such as authentication, billing, or inventory.

Martin Fowler’s Definition

Here’s a concise definition from Martin Fowler and James Lewis.

“In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.” – James Lewis and Martin Fowler

There’s a lot packed into that paragraph. Let’s take a look at four of the main points in this definition.

Loosely Coupled and Loosely Managed

Each service runs in its own process. Many web applications used to run as a single server. So, services running in separate processes are an important break from the past. But the Wikipedia definition of microservices says that they can run in the same process. This illustrates how hard it is to nail these definitions down.

We build microservices around business capabilities. Whether or not a service runs in its own process seems like an academic argument. What the code does is where the rubber hits the road. We build microservices around specific business capabilities, and that’s what puts the “micro” in them. The service focuses on a particular responsibility within the broader application. The number of lines of code and the process footprint isn’t what makes them small.

We deploy and run microservices independently. Microservices stand alone, and we can install and run by themselves. Development teams are also independent, so they can develop applications with tools and languages that are distinct from other services in the same application. The teams also use continuous integration and automated testing.

We deploy microservices with limited centralized management. Microservices architectures have limited and decentralized governance. So, development teams have a lot of autonomy. They manage their schedules, and different parts of an application are deployed at different times. Teams select their platforms, languages, and tools. As we’ll see, this is one of the areas it departs significantly from SOA.

What’s Service-Oriented Architecture?

There’s no single definition of SOA. It’s been around for a long time. Several major software vendors embraced it in its heyday, so the architecture tended to have different definitions depending on where you looked. Thirteen years ago, Fowler wrote an article titled ServiceOrientedAmbiguity where he concludes “I think SOA has turned into a semantics-free concept that can join ‘components’ and ‘architecture.”

The Open Group Definition

But there is a definition from The Open Group. This industry group defines the UNIX trademark, the TOGAF architectural standard, and SOA. The SOA standards page is here. You have to create an account before you can download PDF versions of their document. So, we’ll go to the Wayback Machine and get the page where they used to share their SOA definition for free.

The SOA definition is longer than the one I used for microservices. So, I’m only going to provide a couple of excerpts here.

Service orientation is a way of thinking in terms of services and service based development and the outcomes of services.

A service:

Is a logical representation of a repeatable business activity that has a specified outcome (e.g., check customer credit, provide weather data, consolidate drilling reports)

Is self-contained

May be composed of other services

Is a “black box” to consumers of the service

So far, we can see a lot of overlap between microservices and SOA. The first two bullets work with Fowler and Lewis’ definition of microservices.

But, the third is where things start to diverge. Can you build a new service by combining two others and still call it a microservice? Probably not. The combined services are no longer independent. So, we can see where SOA and microservices are not the same things.

Meanwhile, the last point is just good design. A service that requires knowledge of its implementation is an anti-pattern. I’m surprised that item made it past the committee.

SOA and Centralized Governance

Open Group’s architectural style list implies that it contains “must have” features. Let’s look at a few salient points.

Service representation utilizes business descriptions to provide context (i.e., business process, goal, rule, policy, service interface, and service component) and implements services using service orchestration.

It requires strong governance of service representation and implementation.

The standard mentions service orchestration and states that if your architecture doesn’t use it, it’s not SOA. This requirement runs directly against how Fowler and Lewis describe microservices. Service orchestration implies the kind of synchronization and coupling between services that microservices do not have.

They also clearly state that SOA requires strong governance.

So, from just a brief look at a definition of SOA, we see clear differences with microservice architecture.

Microservices Vs. SOA: What Are the Differences?

Centralized Management

Fowler and Lewis dedicate a section of their post to decentralized governance and data management. Microservice teams work independently. They manage their schedules and work with the tools they wish. They maintain control over how they represent business objects and make their own data storage decisions. Fowler presents this as an advantage. When you read the description of how the teams operate, you can’t help but think of an agile program.

The Open Group requires strong governance in SOA. They have a standard for SOA governance here. (PDF here.)

Here’s a quote:

As a discipline, governance has been with us for many years, but with the advent of enterprise SOA, the need has been heightened for organizations to take governance as a discipline more seriously.

The standard implies that we need to tighten oversight for service-oriented architecture, not loosen it. A quick scan of the standard brings to mind daily status meetings, launch checklists, and quarterly software releases.

Independent Services and Service Orchestration

We think of microservices as independent entities. They are loosely coupled, if coupled at all, and employ their own messages and domain logic. Fowler refers to this as smart endpoints and dumb pipes.

The SOA standard specifically mentions service orchestration. If you dig deeper, you find discussions of messaging. Services using a message bus to communicate with each other. Many SOA implementations use an enterprise service bus to coordinate activities between services.

This is where the two architectures have their most significant differences. If SOA services are going to communicate across a shared bus, they need centralized governance, even if it is not necessarily as rigorous as the one spelled out by the Open Group. They have to agree on how to represent business objects and the domain logic that manipulates them.

Microservices Vs. SOA: Why Should I Care?

So, it turns out there’s a difference between these two architectural patterns, and it’s a significant one. Microservices stand alone, with decentralized management. Development teams can work independently, and with the tools they choose. Fowler at least implies a direct connection between agile methodology and microservices.

Meanwhile, an application built with SOA is composed of services that are more closely coordinated than with microservices. This means establishing governance in advance, and often requires that development teams use the same tools.

Is one paradigm better than the other? I don’t think so. While the decentralized management and isolation of microservices are inherently attractive, some problem domains can benefit from services that communicate with each over a backplane.

The question you should really be focused on is, do you know how your system is designed, and why?

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!)

Oracle will release Java 12 in March and it comes with a handful of new features. I’m going to cover them in the next few posts. We’ll start this week with switch expressions. I’ll take a look at how they change how you’ll use the language.

A Basic Class

Let’s start with a simple car class.

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

publicclassCar{

publicenumModel{

Standard,

Deluxe,

Limited

}

privatefinalModel model;

publicCar(Model model){

this.model=model;

}

publicStringgetModel(){

returnmodel;

}

}

Car has an enum that indicates whether it’s a standard, deluxe, or limited model. We’ll going to use this flag to manage the car’s different features.

Standard Switch Statement

So, it makes sense that we’d use a switch statement to manage features.

Let’s set up a method to set the number of cylinders based on the model:

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

publicintgetCylinders(){

intcylinders=0;

switch(model){

caseStandard:

cylinders=4;

break;

caseDeluxe:

cylinders=6;

break;

caseLimited:

cylinders=8;

break;

}

returncylinders;

}

We could write this without the local cylinders variable, of course. I started with an example that uses break statements to demonstrate a typical switch.

Here’s a simplified version:

Java

1

2

3

4

5

6

7

8

9

10

11

12

publicintgetCylinders(){

switch(model){

caseDeluxe:

return6;

caseLimited:

return8;

default:

caseStandard:

return4;

}

}

Without the local variable, we need a default case to be sure a value is returned from the method.

Either way, switch statements are a little noisy.

Java 12 Switch Expressions

Java 12 makes this easier to write. You can use a switch expression:

Java

1

2

3

4

5

6

7

8

publicintgetCylinders(){

returnswitch(model){

caseStandard->4;

caseDeluxe->6;

caseLimited->8;

};

}

As the name implies, this new switch is an expression, not just a statement. It returns a value. So, the question of whether to use a local variable when selecting a case goes away.

It’s also much easier to read since we don’t need breaks, and each case fits on a single line.

But, you can still use multiple lines in a case if you need to. Let’s say you want to add another field to Car and use it to change the result of getCylinders(). You can add a code block inside the case, and use a break to return a value.

Java

1

2

3

4

5

6

7

8

9

10

11

12

13

14

publicintgetCylinders(){

returnswitch(model){

caseStandard->4;

caseDeluxe->{

if(year==1979){

break4;

}else{

break6;

}

}

caseLimited->8;

};

}

Even though this adds a couple of break statements, it’s still cleaner and easier to follow than a switch statement.

Here’s a post I wrote about my other working langauge, C++. You can find the original over on Typemock.

There’s an old expression developers use when someone makes a mistake. They say you “shot yourself in the foot.”

There’s a variation on the joke that describes the experience of shooting yourself in the foot in various programming languages. The descriptions have evolved, and some versions are funnier than others. But the C++ gag has remained the same since I first saw it, back when I was still wrestling with Rogue Wave Tools.h++, and the STL was only an unsubstantiated rumor. Here is one example:

You accidentally create a dozen instances of yourself and shoot them all in the foot. Providing emergency medical assistance is impossible since you can’t tell which are bitwise copies and which are just pointing at others and saying, “That’s me, over there.”

C++ lets you do just about anything. If you can convince the compiler that you’ve written legal code, it will compile it. Of course, this means that you can, well, shoot yourself in the foot. Let’s take a look at some of the pitfalls of C++.

We’ll define a pitfall as a bug that compiles but doesn’t do what you expect. There’s quite of a few of these bugs, but we’ll cover a handful.

Overriding Arguments in Virtual Functions

Let’s start with an example of a C++ pitfall with virtual functions.

Consider two classes. One is a subclass of the other.

C++

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

classFoo

{

public:

virtualvoiddoIt()

{

std::cout<<"Doing foo"<<std::endl;

}

};

classBar:publicFoo

{

public:

virtualvoiddoIt()

{

std::cout<<"Doing bar"<<std::endl;

}

};

Next, we have a mainfunction that accesses the subclass via a pointer to the base.

C++

1

2

3

4

5

intmain(){

Bar bar;

Foo*fooPtr=&bar;

fooPtr->doIt();

}

Now, when we run the program, we see this:

1

Doing bar

That’s what we expect. When we access a subclass via a pointer to its base class, we expect the subclass’ version of a function to be executed.

But we can break this without even trying hard.

Now, let’s add a default argument to Bar’s implementation of doit().

C++

1

2

3

4

5

6

7

8

classBar:publicFoo

{

public:

virtualvoiddoIt(intx=0)

{

std::cout<<"Doing bar"<<std::endl;

}

};

Then, run the program again.

1

Doing foo

Oops! C++ gave us the implementation of doit() we deserved, but not the one we needed. It ran Foo’s version of doIt() because it has no arguments.

This is a contrived example. Most developers wouldn’t overload a method and add a new default argument at the same time.

But, what if we don’t add a default argument, but change an existing one in a subclass?

First, let’s make a few changes to our two classes.

C++

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

classFoo

{

public:

virtualvoiddoIt(intx=1)

{

std::cout<<"Doing foo: "<<x<<std::endl;

}

};

classBar:publicFoo

{

public:

virtualvoiddoIt(intx=0)

{

std::cout<<"Doing bar "<<x<<std::endl;

}

};

Next, run this new version of our test program:

1

Doing bar:1

We got the right method, but the wrong default value.

Well, we did get the right one because the compiler is always correct, even when it’s wrong.

Default parameters are trouble, and you’re best off avoiding them. But if you do need them, remember that they’re part of the function signature and affect how the compiler picks methods.

Virtual Destructors

Smart pointers have made working with C++ easier. There’s no reason to worry about memory management anymore, right?

Not so much. Let’s add destructors to our classes.

C++

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

classFoo

{

public:

~Foo(){

std::cout<<"Destroying a Foo"<<std::endl;

}

virtualvoiddoIt()

{

std::cout<<"Doing foo"<<std::endl;

}

};

classBar:publicFoo

{

public:

~Bar(){

std::cout<<"Destroying a Bar"<<std::endl;

}

virtualvoiddoIt()

{

std::cout<<"Doing bar"<<std::endl;

}

};

Next, let’s allocate a Bar on the heap, use it, and then clean it up with delete.

C++

1

2

3

4

5

6

intmain(){

Bar*bar=newBar;

Foo*fooPtr=bar;

fooPtr->doIt();

delete fooPtr;

}

Now, give it a spin.

1

2

Doing bar

DestroyingaFoo

Since we deleted our Bar instance via a pointer to Foo, and Foo’s constructor isn’t declared as virtual, the compiler called instead of the override. This can lead to leaked memory.

If you plan on using polymorphism, declare your destructors virtual.

So let’s make Foo’s destructor virtual and re-run the code.

1

2

3

Doing bar

DestroyingaBar

DestroyingaFoo

That’s more like it!

Here’s a good rule of thumb: if you plan on subclassing a class, make the destructor virtual. If you don’t, make it protected, so if someone tries to create a subclass later, the compiler will refuse to build the code.

Also, don’t create a subclass if you’re not sure that the base class has a virtual constructor. If in doubt, use composition instead of inheritance.

Deleting Arrays

We need an array of Bars.

C++

1

2

3

4

5

6

7

8

9

Bar*bar1=newBar;

Bar*bar2=newBar;

Bar*bars=newBar[2];

bars[0]=*bar1;

bars[1]=*bar2;

delete bars;

If you’ve been coding with C++ for a while, you might see the error right away. We should delete arrays with delete[], not delete.

This code compiles. If you run a debug build, it may stop with an exception, depending on your platform. A release build may run normally, or it may exit with an error.

Here’s what I got with CLion running in Windows:

Destroying a Bar
Destroying a Foo

Process finished with exit code -1073740940 (0xC0000374)

So, it exited with an error. This bug might not make it past unit tests or integration tests.

We hope.

How do you avoid this? Easy. Use a vector. Problem solved. C++’s primitive arrays are an accident waiting to happen since they act like raw pointers.

Class Members in Initialization Lists

Initialization lists are the preferred way to set up a new class instance’s state.

Class members are initialized in the order they are declared, not the order specified in your initialization list. Since it’s defined first, _length was initialized with the value in _capacity. But _capacity wasn’t initialized yet.

Don’t refer to class members in initialization lists, no matter how neat and concise it looks.

This is another mistake that your IDE and your static analysis tools should warn you about. But the code will still compile. It might even work sometimes.

Calling Virtual Functions in Constructors

Let’s finish up with a constructor pitfall.

First, simplify Foo’s constructor.

C++

1

2

3

4

5

6

7

8

9

10

11

12

13

classFoo

{

public:

Foo(){

status();

}

virtualvoidstatus()

{

std::cout<<"Hi there!"<<std::endl;

}

};

Next, edit Bar so it only overrides the status() method. We don’t need a new constructor.

C++

1

2

3

4

5

6

7

classBar:publicFoo

{

public:

voidstatus(){

std::cout<<"Yo!"<<std::endl;

}

};

What happens when we create a Bar?

1

Hi there!

When status() is called, our type is still Foo. So, its version of the virtual function is called.

Don’t call virtual functions in constructors.

Ignoring Your Tools

We have one more C++ pitfall to look at before we’re done.

Two of our pitfalls required ignoring the signs before we fell into the hole. When we deleted an array with the wrong operator and tried to initialize a member with another uninitialized member, both Visual Studio and CLion warned us. (I’m assuming Eclipse would have too.)

Pay attention to your tools. Run static analysis. Turn on your compiler’s warnings and tell it to treat them as errors. Your feet will thank you.

Watch Your Step

It’s possible to code in C++ without steel-toed shoes and a doctor on standby. Both the language and the tools have come a long way in the past decade, and it’s possible to get C++’s superior performance and write clean code that’s stable and easy to maintain at the same time.

TypeMock’s Isolator++ is one of those next-generation tools. You can use it to quickly put together effective tests for your legacy and your new code. Download a trial for Linux or Windows today. Your toes will thank you.

I mentioned how much I enjoy working with Python last week. Here’s another post where I used it to solve a company sysadmin problem.The original post is over here.

You have log files from two or more applications, and you need to see them together. Viewing the data together in proper sequence will make it easier to correlate events, and listing them side-by-side in windows or tabs isn’t cutting it.

You need to merge log files by timestamps.

But just merging them by timestamp isn’t the only thing you need. Many log files have entries with more than one line, and not all of those lines have timestamps on them.

Merge Log Files by Timestamp

Let’s take a look at the simple case. We have two files from Linux’s syslog daemon. One is the messages file and the other is the crontab log.

Here are four lines from the messages file:

Shell

1

2

3

4

Sep400:00:08ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval120000ms.

Sep400:02:08ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval124910ms.

Sep400:04:13ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval109850ms.

Sep400:06:03ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval112380ms.

Sep400:02:08ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval124910ms.

Sep400:04:13ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval109850ms.

Sep400:06:03ip–10–97–55–50dhclient[2588]:XMT:Solicit on eth0,interval112380ms.

It worked!

Let’s dissect that command.

cat messages.log cron.log |

Cat concatenates files. We used it to send both logs to standard output. In this case, it sent messages.log first and then cron.log.

The pipe | is what it sounds like. It’s a pipe between two programs. It sends the contents of the two files to the next part of the command. As we’ll see below, sort can accept a single filename on the command line. When we want to sort more than one file, we use a pipe to send the files on standard input.

sort –key=2,3 > merge.log

Sort receives the contents of two files and sorts them. Its output goes to the > redirect operator, which creates the new file.

The most important part of this command is –key=2,3. We used this to tell sort to sort its input using two fields and three of the files. For some reason, sort starts counting fields at one instead of zero.

So sort was able to merge the two files using the day of the month and the timestamp.

This is our easy case. These log files both had single line entries, and our dataset was for less than thirty days. So we don’t have to worry about sorting by months.

Let’s look at something that’s a little more complicated.

Merge Log Files With Multiline Entries

Here are a couple of Java application logs that we would like to merge.

When we merge these two files, we want the multiline log message to remain together. So, sort’s numeric sorting won’t work. We need a tool that’s capable of associating the lines without timestamps with the last line that has one.

Unfortunately, no command line tool does this. We’re going to have to write some code.

A Merging Algorithm

Here’s an algorithm for merging log files that have multiline entries.

First, we need to preprocess the log files.

Scan the log file line by line until we reach the end.

If a line has a timestamp, save it and print the last saved line to a new file.

If a line has no timestamp, append it to the saved line, after replacing the new line with a special character

Continue with step #1.

We could do this in memory, but what happens when we’re dealing with huge log files? We’ll save the preprocessed log entries to disk so that this tool will work on huge log files.

After we perform this on both files, we have a new one that is full of single line entries. We’ll use the sort command to sort it for us, rather than reinventing the wheel. Then, we’ll replace the special characters with new lines, and we have a merged log file.

And we’re done!

Let’s do it.

Merge Log Files With Python

We’ll use python. It’s available on all systems, and it’s easy to write a cross-platform tool that manipulates text files. I wrote the code for this article with version 2.7.14. You can find the entire script here on Github.

Argparse gives us a list from the arguments passed to -f and opens the output file for us, as we’ll see below.

Python Regular Expressions

Then we’ll create a regular expression. Let’s take a close look at it since this is what you’ll need to change if your logs are formatted differently.

Here’s the whole expression:

^[^0-90-90-90-9\-0-90-9\-0-90-9]

The expression starts with a caret ^. This means the beginning of a line.

But then we have this: [^ ] with some characters in the middle. Square brackets with a caret at the beginning mean not.

So the expression means “if this is not at the beginning of the line.”

The pattern we’re matching is inside the brackets.

0–90–90–90–9\-0–90–9\-0–90–9

Each 0–9 corresponds to a numeral. Each \- is a dash. So it could be read like this: NNNN-NN-NN. It’s a pattern for the date we see at the beginning of each log entry.

So in English, the expression means “if the line does not begin with a date.”

If you need to process logs with a different format, you’ll need to change this. There’s a guide to python regular expressions here.

Sorting the Results

Now, we’ll start the real work.

Open a temporary file.

Open the first log file.

Join lines with no timestamp to their predecessors, as described above.

Repeat this for each file passed on the command line.

For the third step, we’ll chop the newline ‘\n’ from the end of the last line we saved. Then we’ll add an SOH (‘\1’) character and concatenate the lines. (I could’ve done this in one line, but I spelled it out to make it clear.)

Finally, the result of this code is a file named tmp.log that contains the log files preprocessed to be one line per entry.

Let’s finish the job.

Python

1

2

3

4

5

6

7

8

sorted_logs=check_output(["/usr/bin/sort","--key=1,2","tmp.log"])

os.remove("tmp.log")

lines=sorted_logs.split('\n')

forline inlines:

newline=line.replace('\1','\n')

args.output.write(newline+"\n")

Check_output executes an external command and captures the output.

So we’ll use it to run sort on our temporary file and return the results to us as a string. Then, we’ll remove the temporary file.

We wouldn’t want to capture the result in memory with a large file, but to keep this post short, I cheated. An alternative is to send the output of sort to a file with the -o option and then open that file and remove the special characters.

Next, we’ll split the output on the new lines into an array. Then we’ll process that array and undo the special characters. We’ll write each line to the file opened for us by argparse.

Log Files, Merged

In this tutorial, we covered how to merge log files, looking at a straightforward case and then a more complicated situation. The code for this is available on Github, and you’re free to download and modify it for your individual needs.

Python is a fun language, and I’ve been trying to use it every chance I have. This post was one of those chance. I wrote on guide on using Rollout’s feature flags with Python. You can find the original post here.

You’re doing it right. You’ve set up an array of microservices that offer REST APIs to support an enterprise application. Your architecture is comprised of loosely coupled services that have enabled your team to roll out updates quickly in response to new feature requests and bugs. But when it comes time to make a change, you still find yourself struggling with dependencies. While it’s possible to version your APIs, you want something more granular. A new API still requires a release of both a new service and a new client application.

Wouldn’t it be nice to coordinate a change between client and microservice without rolling out new code?

It is possible. Feature toggles (also known as feature flags) give you that ability and more. They provide a way to control new behavior in an application by enabling or disabling it without deploying a different version. You can install these toggles in services and clients. For example with REST services, you can toggle new behavior for a subset of users or only enable it after deploying a new client application.

So we add a new parameter to our configuration file or the command line, default it to “off,” and flip it “on” when the new client is deployed, right?

No. That’s not a sustainable solution. It’s a hack. Updating a configuration file isn’t necessarily easier than deploying a new configuration, nor should it be. In many systems, a configuration change is a new release. And what happens when the migration to the new feature is done? Does the feature stay in the configuration file? How big will your configuration file get after a few years? Is it possible it gets turned on by mistake someday?

There’s a better way. There’s feature flag management. Instead of using configuration variables that evolve into technical debt, you manage feature toggles from a feature flag management platform. You can integrate them into a strategy for improving your code, enhancing your ability to support your application, and making your customers happier. With a central management platform like Rollout, you can group flags and coordinate a new feature between client and service, based on a variety of factors like application version and client platform.

We’re going to add a feature toggle to a REST service written in Python. We’ll start with a simple flag that’s managed in configuration and then migrate it to Rollout as part of a secure feature flag management system.

This tutorial assumes familiarity with basic Python and a little bit of Flask. You’ll need to have Python 2.x installed on your system, along with PIP.

Setup and a Basic Web Server

First, you’ll set up a Python virtual environment to run the microservice.

If you don’t have virtualenv installed yet, you can install it with PIP.

1

$pip install virtualenv

Next, create a directory and run virtualenv inside it. Then use the virtual environment’s Python to install Flask.

Shell

1

2

3

4

5

6

7

$mkdirrollout

$cdrollout

$virtualenv rollout

Newpython executable inflask/bin/python

Installing setuptools............................done.

Installing pip...................done.

$rollout/bin/pip install flask

Now you have a virtual environment installed with Flask. We’ll be doing all of our development here. You can use any editor or IDE you wish for the following examples. I’ll just show code snippets and the output of most commands.

First, let’s create a simple web server, so we’re sure everything is in working order.

So, create a file name webserver.py and add this code:

Python

1

2

3

4

5

6

7

8

9

10

11

#!rollout/bin/python

fromflask importFlask

app=Flask(name)

@app.route('/')

defindex():

return"Hello, Rollout!"

ifname=='main':

app.run(debug=True)

We created a Flask application and set up a route for the root path. When a browser requests the root document, the server will return a greeting in plain text.

Next, let’s make it executable, so we can run the script.

Shell

1

$chmod+xwebserver.py

Finally, it’s time to run it.

Now point your web browser at localhost and port 5000.

We have a simple web server to work with. Next, we’ll add a REST endpoint and a feature flag.

A Python Feature Toggle

Now, let’s turn this into a REST service and add a simple feature toggle.

First, we’ll define a document that represents a normal day. Our service will return this document in response to a REST request. Next, change the one and only route to return that document when it receives a GET request with no parameters. Finally, edit the URL path to look more like a REST application.

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

#!rollout/bin/python

fromflask importFlask,jsonify

app=Flask(name)

normalday={

'id':1,

'title':u'Meh',

'description':u'Just another day',

'party':False

}

@app.route('/calendar/api/v1.0/whatistoday',methods=['GET'])

defget_tasks():

returnjsonify({'today':normalday})

ifname=='main':

app.run(debug=True)

So, we’ve established calendar/api/v1.0/whatistoday as our endpoint. Let’s hit it from the browser.

Rerun app.py.

Then point your browser at the new URL so we can see what our REST response looks like.

We have a basic REST service! Now, let’s add a feature toggle.

Not every day is just another day. We need a feature for holidays that we can flip when we want to return a different response to our GET request.

First, you’ll need to create a configuration file.

Create a file name calendar.properties with a single section named calendar and a single property named holiday. Start with it set to true.

1

2

[calendar]

holiday=False

Next, add the code to read it to app.py.

Add ConfigParser to the imports.

1

import ConfigParser

And use RawConfigParser to process the file.

Python

1

2

3

config=ConfigParser.RawConfigParser()

config.read('calendar.properties')

holiday=config.getboolean("calendar","holiday")

So, we have a boolean that indicates whether or not it is a holiday.

Now let’s use modify our route, so it uses the configuration property to decide what document to return.

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

normalday={

'id':1,

'title':u'Meh',

'description':u'Just another day',

'party':False

}

specialday={

'id':2,

'title':u'Woo-hoo!',

'description':u'A day to remember!',

'party':True

}

@app.route('/calendar/api/v1.0/whatistoday',methods=['GET'])

defget_tasks():

ifholiday==True:

returnjsonify({'today':specialday})

else:

returnjsonify({'today':normalday})

We have two different days defined, and the rest service will return one of the other depending on the configuration setting.

So, start app.py and refresh your browser.

Since holiday is set to True, we see the holiday JSON object.

Next, stop app.py and edit the configuration file to set holiday to False.

Start the service, and refresh the browser again.

Since we flipped value for holiday to false, we see the message for a normal day again.

We’ve added our first feature flag. The value of holiday in the properties file controls our application’s behavior. We can return the desired JSON object by modifying the configuration and without deploying new code. But, we still need to manage the configuration file and restart the application after we change it.

Python Feature Flag Management

A properties file is not a feature flag management system. It requires modifying a configuration file in production, which is an accident waiting to happen. If we want to avoid editing live configurations, we have to version and distribute configuration files like code so we can track changes.

So let’s look at a better way to manage Python feature toggles with Rollout.

Getting Started With Rollout

Once that’s done, sign in to your account and create a new application.

Be sure to select Python, as shown above.

Next, you’ll see instructions for adding Rollout to your application. Leave this dialog box open while you set up your environment so that you can return to it. You’ll want to use it later to finish setting up your application.

Now you need to add Rox to the virtual environment you’ve been using. Execute rollout/pip install rox from the command line.

Pip will generate a lot of output.

Initialize the API

Next, we need to initialize the SDK with the application key shown above. First, add the import statement import rox.server.rox_server to the top of the file. Then add the call to Rox.setup() with your application key.

Python

1

2

3

4

5

6

7

#!rollout/bin/python

fromflask importFlask,jsonify

fromrox.server.rox_server importRox

app=Flask(name)

Rox.setup("YOUR_APPLICATION_KEY");

We’re going to run this once to initialize Rollout, so the rest of the application can stay the same for now.

Click Next on the installation dialog.

Rollout is waiting for the application to initialize. Run app.py.

The application will bring up the web server, but we’re only interested in initializing Rox. When you see the that the server is ready for requests, check and make sure you see the Success message above.

Creating a Python Feature Toggle

Click Go To App on the success dialog in the Rollout dashboard if you haven’t already. On the left-hand side of the page, there is a Flags menu item. It’s not enabled yet, because you haven’t created any flags.

So you need to add a feature toggle to your application. This is a two-step process. First, define the toggle in your application code. Then run the application, and the new flag is registered in the Rollout dashboard.

We’re going to add the feature toggle code in the main source file to keep things simple. In a production application, you’d break things down into different files for each class.

Feature flags are kept in RoxContainers. Let’s add one to the application.

First, import rox_flag into the application.

Python

1

fromrox.server.flags.rox_flag importRoxFlag

Then, create a container class with a single member. Near the top of the file.

Python

1

2

3

classMyContainer:

def__init__(self):

self.isHoliday=RoxFlag()

This class contains a single member named isHoliday. A RoxFlag holds a feature toggle. You’ll see how to use it in a moment.

Create an instance of the container, and then once the container is created, register it with the Rox API. So right before the call to Rox.setup(), call register with the container. Register accepts a name with it. We’ll call this one Flags.

1

2

3

roxcontainer=MyContainer()

Rox.register('Flags',roxcontainer)

Rox.setup("YOUR_APPLICATION_KEY");

Now, run the application and let it finish initializing. Just as before, stop it when it starts waiting for requests.

Return to the Rollout dashboard, and click on the Flags menu.

The flag has been registered with Rollout. Since we registered the flag in a container named Flags, its name in the dashboard is Flags.isHoliday. Naming containers provides a mechanism for organizing flags (and other variables managed by Rox) in a large application.

Using a Python Feature Toggle

Now we need to wire the toggle up so we can use it to control our application’s behavior.

A RoxFlag has an is_enabled() method that we call to see if the feature is toggled true or false. So, let’s integrate the flag into our code.

Here’s the entire application after adding the code to use the toggle:

Python

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

#!rollout/bin/python

fromflask importFlask,jsonify

fromrox.server.rox_server importRox

fromrox.server.flags.rox_flag importRoxFlag

app=Flask(name)

normalday={

'id':1,

'title':u'Meh',

'description':u'Just another day',

'party':False

}

specialday={

'id':2,

'title':u'Woo-hoo!',

'description':u'A day to remember!',

'party':True

}

classMyContainer:

definit(self):

self.isHoliday=RoxFlag()

roxcontainer=MyContainer()

Rox.register('Flags',roxcontainer)

Rox.setup("YOUR_APPLICATION_KEY");

@app.route('/calendar/api/v1.0/whatistoday',methods=['GET'])

defget_tasks():

ifroxcontainer.isHoliday.is_enabled():

returnjsonify({'today':specialday})

else:

returnjsonify({'today':normalday})

ifname=='main':

app.run(debug=True)

Inside the application route, we replaced the boolean we loaded from the configuration file with isHoliday.is_enabled().

Creating an Experiment

We manage flags by adding them to experiments. An experiment is a scheme for controlling flags (and other variables) in production.

Click on Production in the left-hand side menu and then click Experiments. This will bring up a screen with a Create Experiment button. Click that and then fill out the new experiment window with a name.

And we see a console for setting flags to true, false, or split. Several criteria can set the flag’s value. For this tutorial, we’re going to stick with setting the flag as true or false for all users.

The flag defaults to false or disabled. Let’s run the application and see it in action.

Run app.py.

Next, refresh your web browser.

We see the record for a normal day.

Head back to the experiment and set the flag to true for all users.

Next, restart the Python script.

Then refresh your browser again.

It worked! We can manage our feature toggle from the dashboard.

Refreshing the Flag

The REST service only retrieves isHoliday’s value on startup. If we want to toggle the flag, we still need to restart our application. We’d rather be able to toggle the flag in real-time. Let’s fix that.

We can refresh Rox API toggles with Rox.fetch(). We can call this anytime in our application to get the latest state of our flags. But, this is a Flask application, and Flask wants to control the main thread. We need to set refreshing the feature toggle values up as a background task.

It would also be nice to know if the flags have changed each time we run fetch. So, let’s make two last code updates.

First, let’s pass some configuration options to Rox.setup().

We configure the Rox API’s behavior with the RoxOptions class. We’re going to install a configuration fetched handler. This is a callback that Rox will call when the latest state of our flags is fetched.

We created an instance of the class and passed in a lambda to run when the configuration is fetched. We’ll print a message stating that the fetch completed and whether or not there were any changes.

If you run the application now, you’ll see that is called once, on application startup.

1

Refreshed rollout.Found changes=True

Next, we need to install a function that will periodically run and fetch the flag state.

Since Flask wants to control the application threads, installing a thread before it starts or trying to run in another thread before transfer control to it can cause problems. Our best option is to create a Flask job.

Add this code to the application before you execute app.run.

Python

1

2

3

4

5

6

7

8

9

@app.before_first_request

defactivate_job():

defrun_job():

whileTrue:

Rox.fetch()

time.sleep(3)

thread=Thread(target=run_job)

thread.start()

This code installs a thread that will run Rox.fetch() every three seconds. This thread won’t interfere with Flask’s request handler.

Now, start the application again, and make a few requests. Then head back to the Rollout dashboard, change the flag, and make more requests. You’ll see that the service returns a different result depending on how you set Flags.isHoliday.

The REST service is picking up the requests and changing its behavior. You can toggle your feature flag in real-time.

Start Managing Your Feature Flags Today

First, we built a small REST service using Python and Flask. Then ee added a basic feature flag and then upgraded the application to a managed feature toggle with Rollout. We finished by integrating reading toggles with Flask’s main event loop.

This is only the tip of the iceberg! Rollout can manage configuration flags that hold more than an enabled or disabled state. Experiments can distribute flag values over different users based on values such as application version and device.

Sign up for a Rollout trial today and take control of your feature flags and application strategy.

Maybe you’re new to C++. Perhaps you’ve been working with the language for a while and are looking at unit testing for the first time. Either way, unit testing with C++ often seems to have more problems than it should.

There are significant differences between C++ and “modern” languages like Java, C#, and Python. These newer languages, the ones most often associated with unit testing, have different tools. Even their editors, debuggers, and test frameworks are designed differently. But, there’s no reason not to be writing tests for your C++ code.

Let’s discuss some of the obstacles to writing tests for C++. You can conquer these problems and make your code more reliable.

C++ Code Is Complex

C++ code can be complex, and this complexity is precisely why the language attracts some developers. Used effectively, C++ is a fast, flexible, and powerful platform for high-performance applications. Used poorly, it’s a tool for creating untestable legacy code that everyone is afraid to touch lest they lose their weekend.

Compile Vs. Run Time Type Checking

Even though C++ is a statically-typed language, its type system is weak. Type checking is done by the compiler, unlike Java and C#. If you can fool the compiler, you can get away with anything.

With reinterpret_cast we can coerce any pointer type to another with few limitations. With C-style casts, you can make things even worse with an operator that’s harder to spot. So, if you want to write a test for codes that are using these operators, you have to figure out what the original author was trying to do.

The C preprocessor generates code with no type checking at all. Some developers use it to create a private meta language that’s not only not typesafe, but unreadable. Macros can act as an overlaid scripting language that obfuscates the C++ code.

Some consider templates the modern-day replacement for the legacy preprocessor. They’re not wrong, since replacing text macros with compiled code is always a better alternative. But templates replace the preprocessor’s lack of type checking with “duck typing.” This mechanism can still hide your intentions from the compiler and make an end run around type checking.

Java and C# have runtime type checking and provide generics instead of templates. These features can make the code easier to read and retrofit for tests. They don’t mean that all Java and C# applications are easier to troubleshoot or test. For example, a poorly-written (or missing) hashing algorithm can cause many problems with a generic container.

Memory Management

And then there is C++’s manual memory management. C++ 11 removed much of the burden of managing memory with the official adoption of the smart pointer. Boost has been providing similar tools for even longer. But, many legacy developers didn’t get the memo. Moreover, while smart pointers are powerful, they’re not foolproof.

Many “modern” languages lack these features by design. Some developers see this as a strength. Other see it as a weakness.

Java has references instead of pointers, and the C# pointer type has more restrictions than C++’s. Both languages replace manual memory management with garbage collection.

But it’s possible to leak a reference, and even though references are not pointers, Java still has a NullPointerException. Go figure.

Any code can be complicated, and this is hardly a reason to eschew unit testing. A developer can still abuse Java exceptions to the point of being a substitute for goto. An eager functional programmer can make streams inscrutable.

So, while the creators of Java and C# designed them to reduce complexity, it’s still possible to write untestable code. Don’t blame the language.

Write tests now, and use them to increase your knowledge of that complicated legacy system. The tests will help you unravel the design or create a replacement.

Slow Builds

The C++ build process is more involved and more time-consuming than for other languages. The C++ build process has two steps: compiling and linking. That difference alone is significant. But, the time to compile code that makes heavy use of templates and can add even more time.

Add to this how many teams structure their large projects. They create a single target, and that is the product of many other builds. Finally, after those targets finish their builds, the system generates a set of executables. One of them is the test binary.

Toss in build targets for a few different platforms, and a complete build cycle can take many minutes, or even hours. This more involved compiling and linking process slows the code/build/test cycle to a crawl.

So, if tests are difficult to run, developers will run them infrequently. They may even ignore them. Eventually, they’re forgotten.

The solution to this problem isn’t easy, but it’s better than not running tests at all. Break the application down into independent components. Create dynamically-linked libraries and built and test them in isolation.

This is a lot of work, and it may feel like that time could be better spent writing new code. But working with a slow build process that everybody hates is an obstacle. It hampers many aspects of the development process, especially tests.

Mocking Libraries

C++’s differing architecture has more disadvantages than the potential to slow the development process to a crawl.

Interpreted languages, including those that run in virtual machines, are easier to mock and fake. Mocking libraries for these languages have access to private class members. They can also mock concrete classes and functions.

Many C++ mocking libraries lack this ability and are limited to mocking only public and virtual class methods. Instead of mocking concrete methods or classes, they expect you to design your code for testing. Legacy code is often impossible to test without either changing it or writing new code just for tests.

But this isn’t a universal problem with C++ mocking frameworks. Isolator++ addresses these limitations and offers the same level of functionality for its C++ version as it does for it’s .NET edition.

Many Resources Are for Other Languages

A significant problem unit testing C++ is that most of the online resources for writing tests are geared toward other languages. Go to your favorite search engine and enter a query for a topic about testing, and most of the articles you find are for Java, Ruby, or C#.

While unit testing has been around for a long time, the modern implementation we’re most familiar with came to prominence with Extreme Programming (XP) and its cousin, Test-driven development (TDD). We usually associate these movements with Java and Ruby, even though we can apply the core concepts to other languages.

But the core concepts behind unit testing apply to any language. If you’re up to writing clean C++ code, you can use the advice in an article about testing with .NET to your code too.

Testing objects and functions in isolation is a universal concept. Writing tests with Arrange, Act, and Assert, is possible with any language, not only Java and C#. There’s no reason to let the dearth of C++ unit testing resources stop you from testing your code.

No More Excuses

C++ is a powerful language, but as the man in the funny red and blue suit says, with great power comes great responsibility. There is no excuse for writing or maintaining code without tests.

Typemock’s Isolator++ is designed to make adding tests to your C++ code easy, regardless of whether it’s new code that you’ve written today or legacy code from years ago. It can isolate any class for testing, regardless of how its dependencies are designed.

Here’s another post I wrote for Scalyr last summer. I enjoy working with Ruby, and putting together this tutorial was a lot of fun. I hope you enjoy it too!

Let’s continue our ongoing series on getting starting with loggers for different languages and platforms. Back in March, we covered logging with Ruby; now it’s time to take a look at the platform most often associated with that language, Rails.

We’ll start with a simple application with scaffolding for CRUD operations on a single record. We’ll look at Rails’ default logging configuration and how to use logging in an application. Then we’ll look at how logging can be improved and why you might want to improve it.

This tutorial uses Ruby v2.5.1 and Rails 5.2.0. You’ll need to have them installed to follow along. These instructions will use the command line to create and configure the application and will not rely on a specific IDE or editor. We’ll let Rails use SQLite for the backend database.

Create a Rails Application

We’ll start off by creating a Rails application. Run this command first.

Shell

1

2

3

4

5

$rails newlogging_app

create

create README.md

....

$

After few screens of output, Rails has created the application.

Next, we’ll let Rails create a model and a controller for us so we have something interesting to log. Move into the logging_app directory and run this.

Shell

1

2

3

4

5

$cdlogging_app/

$rails generate scaffold User name:stringemail:string

Running via Spring preloader inprocess60139

...

$

Rails creates the scaffolding for a user record with a name and an email address.

Rails has a reputation for making life easy for developers, and logging is an example of why. Let’s look at how we can customize Rails Logger.

What Is Application Logging?

Before we delve into how to improve the Rails logger, let’s go back to first principles. What are we trying to do? What is application logging? We’ve got a good definition in an earlier post in this series.

We’ve already covered that in Rails. We have a way to record information about our application to a file and to standard output, which our DevOps can redirect to where they wish with standard output. But we can do more.

What Is Application Logging for?

Why do we write logs? It seems like an obvious question, but it’s worth considering.

We write logs so we have a record of what’s happening.

We’ve released our code to production, and we need to get to work on something else. But at some point, something will go wrong, and we’ll be asked to diagnose the problem. Logs are often the only thing we have.

We may also be asked for information about what’s happening in production when nothing is wrong. How many pages are loaded per day? Per hour? Per minute? Where are users coming from? What features are used most often, and what features are ignored? While some of this information is available from web analytics, logs can drill down to a higher level of detail.

Application logging is a critical tool for helping us locate and fix errors and for collecting information that can help us improve our applications.

What Information Should We Log?

The messages we’ve seen from Rails are messages about what is happening in the application. They lack context, though. Let’s define what we want from a log message and then configure the system to give us that.

A log entry represents an event. Something happened to the application, and the developer thought it was interesting enough to save for later review.

The most obvious thing we might want to know about an event is when it happened. None of the log entries we’ve seen so far have a timestamp on them. We’ll need to add that.

It’s also useful to have some context. We added this log message to our code earlier.

What happened to debug? We called the logger at debug level, but that information was not passed on. All of the log entries appear as equivalent with the default configuration, reducing their usefulness.

Let’s get started at changing the Rails logger’s configuration.

Configuring Logs

Like most loggers, ActiveSupport::Logger supports different levels. Those levels are debug, info, warn, error, and fatal.

We can log a message at any one of these levels by calling the associated method:

Ruby

1

logger.fatal"This is very bad: "+e.message

This gives us a way to categorize messages, and we’ll add the level to the messages below. But let’s look at filtering messages first.

Debug messages are useful for, well, debugging. Their very name implies they’re for development or for isolating a problem. We don’t typically want to see them in production where they can reduce the signal-to-noise ratio in log files and potentially compromise performance.

We can filter which messages make it to the console or log file in Rails configuration.

If we add this to our configuration, only messages logged at error or fatal will be logged.

Ruby

1

config.log_level=:error

Rails has a hierarchal configurations scheme. We can add logging configuration in config/application.rb or config/environment/{environment_name}.rb. The environment-specific file takes precedence.

So when we add this to config/application.rb:

Ruby

1

config.log_level=:error

And this to config/environment/development.rb:

Ruby

1

config.log_level=:debug

We still see debug messages in development, but only error and fatal in production.

We can change the name and location of log files by supplying Rail with logger instance initialized with our preferences.

Ruby

1

config.Logger=Logger.new("/var/log/logging_app/messages.log")

This also means we can create new subclasses of Logger if we want to change logging behavior.

But let’s change the behavior we’re most interest in right now: log message formatting.

Log Message Format

Rails configuration has a mechanism for supplying Rails with a log message formatter instance. The first step is creating that message formatter class.

We extended the ActiveSupport::Logger::SimpleFormatter and overloaded its call method. We need an instance of SimpleFormatter for the configuration.

Logger passes call four parameters. They are the log level (as severity), the time of the message, program name, and the message. We formatted the severity into a fixed width field and the time as a human-readable string. Then we put the parts together into a familiar message format.

There are a variety of ways we can add this class to our code. The use of the libs directory can be controversial with Rails, but we’ll use it for the sake of getting a formatter installed.

First, create the formatter class in logging_app/libs as message_formatter.rb.

Next, since Rails 5.x does not load classes from lib automatically, add these lines to config/application.rb in the LoggingApp class declaration.

Ruby

1

2

config.autoload_paths<<Rails.root.join("lib")

config.eager_load_paths<<Rails.root.join("lib")

Finally, add the formatter to config/environment/development.rb

Ruby

1

config.log_formatter=MessageFormatter.new

When we run the server and open a user entry, we see log messages with our custom format.

Unit testing is a subject that’s near and dear to my heart. Here’s a post I wrote for TypeMock about that very thing. The original is over here.

—

Why isn’t unit testing with C++ as common as it is with Java, C#, and Python?

I don’t know. I was hoping you did.

If you’re developing with C++, you should already be writing tests. There’s no reason not to, and many reasons why you need to start today.

What Is Unit Testing?

Before I discuss the why and how of unit testing with C++, let’s define what we’re talking about.

Unit testing means writing code that verifies individual parts, or units, of an application or library. A unit is the smallest testable part of an application. Unit tests assess code in isolation.

In C++ this means writing tests for methods or functions. Tests only examine code within a single object. They don’t rely on external resources such as databases, web servers, or message brokers.

For the sake of this discussion, I’ll assume that you execute your unit tests as part of a build or, even better, a continuous integration system.

Why Unit Test With C++?

Find Stupid Bugs Early

We all write stupid bugs. We create off-by-one errors, define incorrect constants, mess up an algorithm, or write one of those “what the heck was I thinking?” mistakes every day. We’re only human. Even us C++ developers, despite what those C# folks whisper in the lunchroom.

Without unit tests, we don’t catch these errors until we get to integration testing or worse, QA. But it’s just as likely that we won’t find them until later in production.

So, unit tests act as a safety net. By taking the time to write checks for each method or function as we write them, we catch stupid bugs when we create them.

Avoid Regressions

Not all bugs are stupid. Some bugs are quite intelligent. We call them regressions. If you’ve been around for more than a few years, you’ve seen one of these.

Your system has been working in production for a long time, but you need to add a new feature or address a deficiency. So you modify the code and roll out a new version, and something else breaks. If you’re lucky, it’s easy to figure out. If not, cancel your weekend plans.

Your first line of defense against regressions is unit tests. Creating targeted routines that verify discrete behavior, and then running them on every build helps ensure that a new feature doesn’t come with a new bug.

Get Early Feedback

Unit testing with C++ gives you early feedback. Sometimes the feedback is that you haven’t caused a new regression. Other times it’s whether your new code is doing what you think.

Early feedback means confidence. Without unit tests, you don’t know if what you’re doing works. Writing code that compiles the first time feels great. Writing code that does what you want right away feels even better.

Better Design

Writing unit tests for code means writing code that can be broken down into discrete units. Testable code is modular because discretely tested units are loosely coupled.

If this reason for unit testing, combined with the previous one, reads like an advertisement for test-driven development (TDD), that’s because they are. TDD works and it’s a reason to start using unit testing with C++. Agile systems are associated with languages like Java, Ruby, and C#. Is that because of the languages or the practices associated with them?

No. There’s no reason you can’t implement a rapidly evolving architecture with C++.

Create Built-in Documentation

Raise your hand if you trust comments.

Yeah, I didn’t think so. Even your comments can be unintelligible or outright wrong after a few months.

But have you ever peeked at unit tests to see how something works? Do you breathe a sigh of relief when you check out code and find tests? Tests don’t lie. Sometimes the best way to figure out how the code works is to see it in action.

How to Unit Test With C++

Okay, so you’re sold on unit testing now, right? Then let’s get to it.

Pick a Test Runner

Unit testing with C++ requires a test runner. We write tests as functions, and then we link the functions into a binary that the build executes as a test target. So, we need a main function that knows how to run the tests, check the results, and print a report.

There are too many test runners and frameworks for unit testing C++ to list here. I’ll list a few of the more common ones.

Google Test is the most well-known cross-platform test runner for C++. It’s distributed as source code. So, you have to build it for your environment or include it as a vendor dependency with CMake. It comes with a mocking library and, unlike Microsoft’s runner, is open-source software.

Pick a Mocking Framework

Now you can run tests. So, you need to write them. The key to writing effective tests is to pick a discrete unit and then verify its functionality in isolation. If a test fails and it’s not obvious what caused the failure, the unit under test is not isolated.

Depending on what we are testing, we may need to use a mock. Mocking is creating an object that mimics the behavior of another object. For example, if we are testing an object that interacts with a messaging API, we would mock the messaging connection, rather than write a test that requires connectivity with a messaging broker.

Typemock’s Isolator++ is a C++ mocking library for Windows and Linux. With it, you can fake any class or template and any method, without changing your existing code.

The Google Test project bundles Google Mock with the test runner. You can use it to mock C++ classes and templates but the library has limitations that make working with concrete and free functions difficult.

You can read a comparison of Isolator++ and Google Mock here.

Use Dependency Injection

If you are writing new code, or able to change legacy code, dependency injection (DI) is your friend. While you may only associate DI with Java, it’s available in C++, too. There’s even a Boost library in the works.

But you don’t need a framework, to use DI. If you don’t want to use an experimental library or move to a new version of C++ yet, you can still use DI in your C++ code.

The most basic form of DI is constructor injection. Fowler describes it in the article I linked above using a framework for Java, but it’s easy enough to roll your own if you don’t want to add a new external dependency.

DI means separating the creation of an object from its use. This pattern makes it easy to replace the implementation of a service with a new one. Fowler even uses the term plugin to describes injected objects in his post.

But DI makes testing your code easier too. You can pass a fake to an object on initialization and then use it to observe the object under test. Dependency injection makes isolating objects for verification easy.

If you are working with complicated and tightly coupled legacy code, a mocking framework like Isolator++ can help you create tests. The framework is ideal for adding tests to legacy code. But writing your code with a pattern like DI will always make things easier and refactoring that legacy code might be a good idea too.

Write AAA Tests

Tests that are hard to decipher are almost as bad as no tests at all. (But only almost.)

Like clean code, there’s such a thing as clean tests. Clean tests are easy to read and focus on a single piece of behavior.

One way to make sure your tests are easy to comprehend is implementing the three A’s: Arrange, Act, and Assert. If you structure your tests with this in mind, they will be coherent and comprehensive.

Arrange gathers the test requirements and prepares them for the test. It’s where you create the object to test and set any preconditions it needs. It’s also where you create your mocks.

Act is the test operation itself. It’s usually the short part of the test. Ideally, it’s where you call a single function or method.

Assert verifies that the test succeeded. It’s where you might see a list of assert statements that check the state of different variables.

The Mandate of Unit Testing With C++

There’s no reason to write C++ code without unit tests. The patterns and practices associated with testing are all possible with the language because, as we all know, there is nothing you can do with another language that you can’t with C++. Most other languages are implemented in C++ under the covers.

So, get started today! Isolator++ supports C++ on Windows and Linux and contains features that will help get you started with unit testing right away. Download a trial license today.