Session 412WWDC 2016

Xcode 8 makes it easier to find several new categories of bugs with improvements in Runtime Sanitization and the Clang Static Analyzer. The Thread Sanitizer will help you find data races and other concurrency bugs. The static analyzer has been extended to search for localizability issues, check nullability, and find memory leaks in MRR code.

[ Music ]

[ Applause ]

Hello, I'm Anna.

Welcome to the Thread Sanitizer and Static Analysis talk.

Since our team works on bug-finding tools, we are goingto tell you about new ways of catching bugs.

I'm going to start with giving a brief overviewof Address Sanitizer and then dive much deeperinto Thread Sanitizer which is a new feature we introducingthis year.

Later, Devin is going to come up and tell youabout the new checks we've added to the Clang Static Analyzer.

They work with Swift and Objective-C,and they have tight integration into the Xcode UI.

So last year we've introduced Address Sanitizerto macOS and iOS.

This tool finds memory corruptions such as stackand heap buffer overflows use up the freeze, double freeze.

It's extremely effective at finding memory issues.

So if you're not using it already, I highly,highly, highly recommend it.

This year we've extended the toolto provide full support for Swift.

Which will be especially exciting to those of youwho love to live dangerously in Swift.

So what does it mean if you are using unsafe pointer types?

Run your test with Address Sanitizer turned on,it will find some bugs for you.

Now, while Address Sanitizer mainly focuseson memory corruption issues, there is another large sourceof bugs that are threading issues.

These are even harder to reproduce and debug.

They're sensitive to timing.

They might occur only in some certain circumstanceswhich means that the appellationsthat contain them will have unpredictable behaviors.

So this year we introduce support to another tool,Thread Sanitizer which will help you to both findand better understand your threading bugs.

TSan reports mainly different kinds of bugs,so let's take a look at some of them.

It will tell you about use of uninitialized mutexes.

This might not seem like a big deal.

However, if you are using a mutexthat is not appropriately initialized that will leadto very subtle bugs in your applicationsbecause you're not actually getting any mutual exclusionwhen you use such a mutex.

Another example are thread leaks.

If your application has a lot of threadsand if those threads are leaked, you will,and if there's memory leaks.

Another one unsafe call in signal handlersand unlocks from a wrong thread.

However, data races are by far the most common problembecause they're so easy to introduce.

They happen when multiple threads access the same memorylocation without using proper synchronization.

So let's see how this tool works by going into an Xcode demo.

So here I'm going to demo this Thread Sanitizeron an alpha version of last year's WWDC app.

So here as you would expect,it brings up a schedule for the week.

However, notice this interesting visual bug.

Even though all the session's data has been downloaded,the network activity indicator keeps spinning.

Now, I know I use a global variable to decide when to showand hide this indicator so there might be a threading problem.

Let's see if Thread Sanitizer can help us find it.

In order to turn on Thread Sanitizer, we go to edit scheme.

Choose diagnostics tab.

And click here on enable Thread Sanitizer.

Now, here you can choose to pause in the debuggeron every single issue and debug that issue right there.

Or you could choose to keep running,collect all the threading issues that Thread Sanitizer report,and explore them later on.

The second workflow is new in Xcode 8, and it's only supportedby Thread Sanitizer, so let's take a look how that works.

When your launch application under Thread Sanitizer,Xcode is going to rebuild your projectwith extra compiler instrumentation, and it's goingto launch it in a special mode that triesto find threading issues.

So here is our application is up.

And Xcode tells us that Thread Sanitizer detected two issuesby displaying this purple indicatorin the activity viewer.

Clicking on this purple indicator will take usto the issue navigator.

And while previously we only used itto display build time issues such as compiler warnings,compiler errors, Static Analyzer issues.

This year has been extendedto provide support to runtime issues.

And this is where Thread Sanitizer's issue foundits home.

So Thread Sanitizer reported two problems.

Let's take a look at each one.

The first one is use of uninitialized mutex.

Now, this problem occurred as you were runningthat application sometime in the past.

Thread Sanitizer is going to tell us about that exact momentby providing a historical stack trace.

Even though this is not a live stack trace,you can walk its frames as if it was a live stack trace.

So let's take a look.

At some point we called acquire lock.

That called pthread mutex lock,and passed an invalid mutex reference.

That was called from reset feed statuswhich was called from the initializer.

Now, as you can see here we do initialize the mutex,but we initialize it after we use it.

It's a simple ordering bug.

So just reordering thoseto statement should take care of that.

Okay, let's go on to the second problem which is a data race.

Also, here Thread Sanitizer tells that there is a data raceon the variable called activity count.

Now, that's the same global variable that they use to decidewhen to show and hide that indicator.

Since this is a data race,Thread Sanitizer will tell us about two events.

The two race accesses.

A read and a write here.

So the read happened on thread 11,and the write happened on thread 13.

Notice that neither of those are a main thread,and the stack traces are the same which meansthat they are probably executing the same cordfrom multiple threads without using synchronization.

So let's take a look.

Okay, here we are updating this activity account variable.

Now, I could have fixed this race by adding a lock.

But notice that this is just a symptom.

The next line here updates the UI.

And we know that the UI updates should happenon the main thread.

So the proper fix here isto dispatch both the counter increment and the UI updateonto the main cue with Grand Central Dispatch.

This will both take care of the logical problemin our application and also take care of the racebecause all the threads will accessthat count variable from the same thread.

Now, I'm sure I sound very convincingand you all believe me that I fixed the bugs.

However, the best way of checking yourself isto run the tool again on your project.

So we should rerun the application againwith Thread Sanitizer turned on.

And again it's going to rebuild your projectwith this extra checking and launch it in the special mode.

Now, the application is up.

We see that the strange visual UI bug is gone,and Thread Sanitizer doesn't report any issues.

So all is well.

Let's go back to slides.

[ Applause ]

So just to recap the demo, you can enable Thread Sanitizerin the Scheme Editor when you go to the diagnostics tab justlike you did with Address Sanitizer.

In addition to ASan's workflow of stopping in the debuggeron the very first issue,Thread Sanitizer it supports an additional mode.

Where you could keep routing as the issues that detected,and then you could explore them in the issue navigator.

They will stay there until you launch an application again.

So let's now talk about what Xcode does behind the scenesto make this all work.

In order to use Thread Sanitizer,Xcode passes a special flag to both Clang and Swift compilersthat instruct them to produce an instrumented binary.

This binary links to a TSan runtime library that is usedby the instrumentation to both monitor the executionof the program and detect those threading issues.

So if you're building and running Cocoa command line,you can pass an option to either of the compilers.

And Xcode will also support Thread Sanitizerby providing enableThreadSanitizer option.

Now by default, TSan will keep runningas the errors are detected.

But you can instruct it to abort on the very first issueby setting this TSan options environment variables to halton error equals 1 when you launch your process.

That will allow you to have the same workflow as what you havewith Address Sanitizer.

So where can you use this tool?

Thread Sanitizer is supported on macOSand in the 64-bit simulators.

It is not supported on the device.

So now you know how to use this tool,how to launch it, how to find issues.

Let's talk about what, how you can fix the bugs it reports.

And we'll focus mainly on data racesbecause this is the biggest category of bugs it reports.

So what is a data race?

Data race happens when multiple threads access the same memorylocation without using proper synchronization and whenat least one of those accesses is a write.

And the problem here that you might not only endup with stale data, but the behavior here is unpredictable.

You might even end up with a memory corruption.

So what are the reasons for data races?

Well, it often indicates that you have a logical problemin the structure of your program.

And only you will know how to fix it.

On the other hand, it also meansthat we are missing some synchronization.

So let's talk about that second scenario.

Here is an example of a data race in Swift.

We have a global variable data.

We have a producer that sets it for 42and a consumer that prints it.

If those two pieces of code are executedby two different threads, there will be a data race.

So how about this code?

We introduce another variable called is data available.

And we set that flag after we update the data in the producer,and in the consumer we are going to wait until the flag is setand then if once it's set, we print the data.

Well, this looks very logical.

It seems like it should work.

The problem here is what you see is not what will get executed.

The instructions here can be reordered by either the compileror the CPU, so you cannot assume that the flag is setafter the data is updated.

The order of the instruction is not guaranteed neitherin the producer nor the consumer.

So what is the point here of this slide?

I just want to demonstrate that tryingto roll your own synchronization methods is often not agood idea.

What should we do instead?

We should use something that's available already.

For example, Grand Central Dispatch is a very good option.

You can dispatch the recent accessesonto the same serial queue that will make surethat they execute it on the same thread,and there will be no data race.

So now as you might recall Thread Sanitizer worksfor both Objective-C and Swift.

So let's use Objective-C for our next example.

Here is lazy initialization code.

And we are implementing method called getSingleton.

That makes sure that we return the same shared instanceto all of its callers.

Now, if this code is executed by multiple threadswithout proper synchronization, there will be a data racewhen both threads try to update the shared instance variable.

Okay, so what about this code?

We tried to fix the problem by allocatingand initializing a local variable,and then we are using atomic compare and set operationto make sure that threads atomically updatethat global variable.

So there will be no data race on the right.

This might look like a step in the right direction,but this code still has problems.

So let's take a look at them.

First, it's very difficult to reason about memory managementwhen you are using atomics.

So, for example, here you will have use-after-freeif you are using ARC.

And if you are using MRR, this object will be leaked onlyin case there is a race.

So that's not good.

That's not the only problem.

Another problem here isthat since the read is unsynchronized,there could still be a race where one thread is tryingto read that shared variableand another one is trying to atomically set it.

So this is undefined behavior, and that's not good.

What should you do instead?

I mean, if you know the solution already,use Grand Central Dispatch.

Dispatch wants performed laziness socialization for you.

It's even simpler in Swift.

Both global variables and class constants have dispatchone semantics.

So you can choose either of those two solutions,whatever works best for your code.

Okay, so just to summarize,you should use the highest level API that's suitableto your needs.

And most people should be using Grand Central Dispatch.

If that's not suitable, you can use pthread APIs or NSLock,for example, now, we do have a new OS unfair lock that's newon our platform this year, and it replaces OSSpinLock.

And they also have C++ and C11 Atomics.

They are supported by Thread Sanitizer.

But as you've seen in the previous example,they're very difficult to use correctly.

And besides the performance,being here is either not measurable or negligible.

So don't choose to use those APIs if you did not measurethat they actually have something on your application.

So for more information about all of those APIs,please attend Concurrent Programming talk on Friday.

So now let's talk about benign races.

What are those?

Some developers argue that on some architectures,for example x86, you do not need to insert synchronizationbetween a read and a writebecause the architecture itself guarantees automaticityof those operations on pointer size data.

Now, it's important to remember that any race,even at a benign race, is consideredto be undefined behavior from C or C++ standard.

So not only will you be surprised if that codewith benign races write, runson an architecture you have not tested well on before.

But the compiler is free to reorder those instructionsas if no other thread saw that.

So the bottom line is that you might endup with very subtle bugs.

So as our engineering leadfor Thread Sanitizer [inaudible] "Fix all the bugs."

[ Applause ]

Now, to the most exciting part of our talk.

As we all know, data races are hard to reproduce since they'reso sensitive to timing.

So the most interesting thing about Thread Sanitizer isthat it can detect races that did not even manifest duringthat particular program run.

Let's see how it does that.

When you compile your program with Thread Sanitizer,it instruments every memory access, and it prefixes itwith a check, with a code.

But first, records the information about that access.

And second checks if that access participates in a race.

So let's take a closer look.

For every aligned 8 bytes of application memory,Thread Sanitizer shared those state,keeps track up to four accesses.

If you're interested in more information,you can go to your session website.

And there's also several related sessionsthat we think you might find helpful.

Thank you.

[ Applause ]

Apple, Inc.AAPL1 Infinite LoopCupertinoCA95014US

ASCIIwwdc

Searchable full-text transcripts of WWDC sessions.

An NSHipster Project

Created by normalizing and indexing video transcript files provided for WWDC videos. Check out the app's source code on GitHub for additional implementation details, as well as information about the webservice APIs made available.