Topics

Featured in Development

Understandability is the concept that a system should be presented so that an engineer can easily comprehend it. The more understandable a system is, the easier it will be for engineers to change it in a predictable and safe manner. A system is understandable if it meets the following criteria: complete, concise, clear, and organized.

Featured in Architecture & Design

Sonali Sharma and Shriya Arora describe how Netflix solved a complex join of two high-volume event streams using Flink. They also talk about managing out of order events and processing late arriving data, exploring keyed state for maintaining large state, fault tolerance of a stateful application, strategies for failure recovery, data validation batch vs streaming, and more.

Featured in Culture & Methods

Tim Cochran presents research gathered from ThoughtWorks' varied clients and projects, and shows some of the metrics their teams have identified as guides to creating the platform and the culture for high performing teams.

This repository is Google’s current implementation of TCMalloc, used by ~all of our C++ programs in production. The code is limited to the memory allocator implementation itself.

As hinted above, TCMalloc includes implementations for the C *alloc family and for C++ ::operator new and ::operator delete. These provide a number of optimizations over their respective counterparts that come with the C and C++ standard libraries. For example, TCMalloc performs allocations from the OS using fixed-size "pages", which simplifies bookkeeping. Additionally, some of those pages are dedicated to objects of specific sizes, e.g., all 16-bytes objects. This also brings a simplification when it is time to get or release that memory. Finally, commonly-used objects are cached for speed of operation.

TCMalloc also supports telemetry extensions via MallocExtension which can be useful to gather heap profiles and snapshots to investigate memory behaviour.

A number of configuration options are available to tune TCMalloc performance. In particular, you can define the logical page size, which can be 4KiB, 8KiB, 32KiB, or 256KiB. Larger page-sizes will reduce the probability of requiring a new page allocation from the OS, thus speeding up operation at the cost of larger memory consumption. It is also possible to set the cache size on a per-thread or per-CPU basis, which is the default. Similarly to page sizes, larger cache sizes will improve performance. Finally, you can tune how aggressively memory is released, which also affects performances in several ways.

The following diagram shows TCMalloc architecture, which is thoroughly described in the relevant document:

TCMalloc can only be built using Bazel, Google's internal build system, and this might come as a less-than-ideal surprise to some developers using other build systems. Bazel, though, is available in binary format for macOS, Ubuntu, Fedora, and Windows, so this should not really be a major hindrance.

They say it can be built with autoconf/automake/libtool

"As of 2.1 gperftools does not have configure and other autotoolsproducts checked into it's source repository. This is common practicefor projects using autotools.

NOTE: Source releases (.tar.gz that you download fromcode.google.com/p/gperftools) still have all required files just asbefore. Nothing has changed w.r.t. building from .tar.gz releases.

But, in order to build gperftools checked out from subversionrepository you need to have autoconf, automake and libtoolinstalled. And before running ./configure you have to generate it (anda bunch of other files) by running ./autogen.sh script. That scriptwill take care of calling correct autotools programs in correct order.

If you're maintainer then it's business as usual too. Just run makedist (or, preferably, make distcheck) and it'll produce .tar.gz or.tar.bz2 with all autotools magic already included. So that users canbuild our software without having autotools."

Re: They say it can be built with autoconf/automake/libtool

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Juan,

gperftools is a distinct project. It contains the older allocator Google released a few years back. The newer one is TCMalloc (github.com/google/tcmalloc). I think this should explain what you are reading.