In Part 1, we started looking at some benchmarks regarding whether Protobuf is actually the better choice as opposed to JSON for performance reasons. Let's continue the discussion here.

Decode Object

We have proved JSON is slow for numeric input. What about object binding itself? Is the benchmarking result bad because binding is slow in JSON? Given that we are using 10 fields in the benchmark, let’s find out.

To make the game fair, we use a short and pure ASCII string field this time. The string copying performance should be very similar. So, the performance difference should come from the binding process.

message PbTestObject {
string field1 = 1;
}

library

compared with Jackson

ns/op

Protobuf

2.52

68666.658

Thrift

2.74

63139.324

Jsoniter

5.78

29887.361

DSL-Json

5.32

32458.030

Jackson

1

172747.146

For 1 string field, Protobuf is actually slower than Jsoniter by 2.3x.

We can repeat the same test for 5 fields, 10 fields, and 15 fields to see a pattern.

Hash will collide, so use with caution. If the input is likely to contain an unknown field, use the slower version to check the string again once hash matched. Jsoniter has a decoding mode DYNAMIC_MODE_AND_MATCH_FIELD_STRICTLY, which will generate exact matching code:

Encode Integer List

Integer list encoding should be fast in Protobuf, which does not need to write out all those commas.

library

compared with Jackson

ns/op

Protobuf

1.35

159337.360

Thrift

0.45

472555.572

Jsoniter

1.9

112770.811

DSL-Json

2.19

97998.250

Jackson

1

214409.223

Protobuf is only 1.35x faster than Jackson. Although integer object fields are faster in Protobuf, integer lists are not. There is no special optimization here. DSL-JSON is faster than Jackson because individual numbers are written out faster.

Protobuf is about 1.3x faster than Jackson for object list decoding, but DSL-JSON is faster than Protobuf.

Encode Object List

library

compared with Jackson

ns/op

Protobuf

2.22

328219.768

Thrift

0.38

1885052.964

Jsoniter

3.63

200420.923

DSL-Json

3.87

187964.594

Jackson

1

727582.950

Protobuf is more than 2x faster than Jackson for object list encoding, but DSL-JSON is faster than Protobuf. It seems like Protobuf is not good at list encoding/decoding.

Decode Double Array

Java arrays are special. double[] is more efficient than List<Double>. It is very common to see an array of doubles to represent the value/coordinate of the time interval. However, the Protobuf Java library does not speak double[]. It will always use List<Double>. We can expect a win for JSON here.

message PbTestObject {
repeated double field1 = 1 [packed=true];
}

library

compared with Jackson

ns/op

Protobuf

5.18

207503.316

Thrift

6.12

175678.703

Jsoniter

3.52

305553.080

DSL-Json

2.8

383549.289

Jackson

1

1075423.265

Protobuf is more than 5x faster than Jackson for double array decoding, but compared with Jsoniter, Protobuf is only 1.47x faster. So, if you have a lot of double numbers but they are in the array instead of on the fields, the performance difference is smaller.

Fast Path

for (int i = 0; i < chars.length; i++) {
bb = buffer[ci++];
if (bb == '"') {
currentIndex = ci;
return i;
}
// If we encounter a backslash, which is a beginning of an escape sequence
// or a high bit was set - indicating an UTF-8 encoded multibyte character,
// there is no chance that we can decode the string without instantiating
// a temporary buffer, so quit this loop
if ((bb ^ '\\') < 1) break;
chars[i] = (char) bb;
}

This fast path avoids the cost to process escaped char and UTF-8.

JVM Hotspot Optimization

Before JDK9, java.lang.String was char[]-based. The input is byte[] and UTF-8 encoded; we cannot copy directly from byte[] to char[]. In JDK9, java.lang.String is changed to be byte[]-based. If we take a look the JDK 9 source code:

Using the deprecated but still available constructor, we can use Arrays.copyOfRange to construct a java.lang.String now. However, after testing, it turns out this is not faster than DSL-JSON implementation.

It seems like the JVM Hotspot is doing some loop code pattern matching here. If the loop is written in this way, the string is constructed directly from copying byte[]. Even if in JDK9 with +UseCompactStrings, in theory, the byte[] > char[] > byte[] conversion should be slow. However, it turns out DSL-JSON implementation is still the fastest.

If the input is mostly string. Then this optimization is crucial. The art of parsing in Java is more about the art of copying bytes into a JVM-stupid java.lang.String. In a modern language like Go, the string is UTF-8 byte[]-based, which is wise.

Encode String

Similar problem. We cannot copy char[] into byte[].

library

compared with Jackson

ns/op

Protobuf

0.96

262077.921

Thrift

0.99

252140.935

Jsoniter

1.5

166381.978

DSL-Json

1.38

181008.120

Jackson

1

250431.354

Protobuf is slightly slower than Jackson when encoding long strings. Again, the char[]-based string is the problem here.

Skip Structure

JSON is a format without a header. Without a header, JSON will need to scan every byte to locate the field needed — even if other fields are not intended to be parsed.

The message will be written using PbTestWriteObject and read by PbTestReadObject. field1 and field2should be skipped.

library

compared with Jackson

ns/op

Protobuf

5.05

152194.483

Thrift

5.43

141467.209

Jsoniter

3.75

204704.100

DSL-Json

2.51

305784.845

Jackson

1

768840.597

Protobuf can skip the structure faster than Jackson by 5x. If skipping long string, the cost for JSON will be linear to the string size, while the cost of Protobuf is constant time.

Summary

It is the time to count the scores!

scenario

Protobuf V. Jackson

Protobuf V. Jsoniter

Jsoniter V. Jackson

Decode Integer

8.51

2.64

3.22

Encode Integer

2.9

1.44

2.02

Decode Double

13.75

4.4

3.13

Encode Double

12.71

1.96 (only 6 digits precision)

6.5

Decode Object

1.22

0.6

2.04

Encode Object

1.72

0.67

2.59

Decode Integer List

2.92

0.98

2.97

Encode Integer List

1.35

0.71

1.9

Decode Object List

1.26

0.43

2.91

Encode Object List

2.22

0.61

3.63

Decode Double Array

5.18

1.47

3.52

Encode Double Array

15.63

2.32 (only 6 digits precision)

6.74

Decode String

1.85

0.77

2.4

Encode String

0.96

0.63

1.5

Skip Structure

5.05

1.35

3.75

Worst case scenario for JSON:

Skip very long string — proportional to the string length.

Decode double field — Protobuf can be 4.4x faster.

Encode double field with precision — it can be 12.71x slower to write all the digits out.

Decode integer — Protobuf can be 2.64x faster.

If your real workload is unlike the worst case scenario mentioned above, but mostly composed of strings, then the speedup should be < 2x (compared against Jsoniter); Protobuf can even be slower if you are really unlucky.