Building custom Xcode Instrument packages (XML strikes back)

“Write more than one,” they said…

Not long ago, I started working with custom Instrument packages. I was excited after Apple WWDC session 410 last year and thought it would be great to build something. In my previous article, I described overall package architecture, some of the pitfalls in development workflow, and how in AppSpector, we had made a custom package to track traffic compression in our SDK.

During the 410 sessions, Apple engineers said: “Build more than one instrument” on a separate slide. Custom package for Instruments is a perfect tool for implementing a precise debugging flow. Most developers know how long it can take to fix a bug when you are unclear on what's happening.

You are almost done when you find a "point of view” to look at the problem. The fix is always something like 10%, and finding that correct view angle takes the rest of your time. Thinking of some ideas for a package, I came across a great article by CossackLabsabout the open tracing framework and its implementations.

The idea of visualizing continuous processes inside your app looked like a perfect candidate for being an instrument. Actions performed in the context of different view controllers, operations stacked in queues, and complex processes like a client-backend message exchange, all these cases perfectly match the open tracing ideas.

Package architecture

Let's start with a basic approach: to implement open-tracing-like graphs. I used graph elements, which looked like the best choice. Open tracing framework deals with two fundamental ideas of scope: a context where something is happening; and span - some activity inside a scope.

Scopes within the package implementation have start and stop events, unique names, and the result: success or failure. This is simple compared to what mature open tracing frameworks suggest, but enough to experiment with graphs in the packages and be helpful.

The packaging scheme is simple. I’m not going to dive deep into the elements described in the first part, and I will only tell a bit about graph-specific things.

The first problem I faced when I built a raw implementation during a weekend was that the graph lane appeared in UI only after I sent the stop event. I wanted to display it from the start event and highlight with different colors those spans running at the moment.

Thanks to Kacper, an Apple engineer participating in 410 sessions, I figured out that instruments have a unique structure for this called `open-interval-template.` It’s a structure that defines fields available to the package between the start and the end events.

The graph itself is described with a `graph` node, which contains fields like title and reference to the table from where to get data. `Pplot-template` structure describes graph lanes displayed.

The example below means we’ll have a separate route for each unique scope, and it will get its name from a scope-name variable. The last two tags under `plot-template` describe lines inside the lane. The color will be taken from `status-color` and the label from `span-name.`

Another exciting thing is that the `open-interval-template` can only operate variables that appear in the start pattern, which makes sense because when you display started but not the finished event, your code can’t use data from the end pattern.

Also, in the last two columns of the table, we calculate the color for the line and status label. To do so, we need to know the state of the event -- is it running or already finished? The only way I can resolve this is only to check the variables in the end pattern.

That's why you see `span-name-started` and `span-name-stopped` variables in the start and end patterns. The first is used to draw a span name on a line, and the second is to distinguish running spans from ended. This is one of the ugly things about custom packages; it should have another, more elegant solution.

First attempt

I was excited to try tracer on actual code, so I looked for a good example. The first try was AppSpector SDK and message processing module. Each message, incoming and outgoing, passes through several steps: serializing, packaging, compression, and finally, sending. Sometimes messages could wait in a queue for different reasons like network failures.

This looked like a perfect candidate, so I inserted tracer calls where appropriate and started recording, expecting a picture as in an open tracing manual with many overlapping lanes. But the actual result was a bit different:

Span duration matters. Unfortunately, if you have milliseconds duration spans with seconds intervals between them, Instruments scales graphs, and all you see are some green dots. Only zooming allows us to see actual spans:

It would be great to have the ability to make the timeline nonlinear so that seconds passed between tracked events will occupy less space than millisecond-long action.

Operations

Looking for the following example, I decided to try BitBot, the Bitrise CI client I wrote about not long ago. It has lots of NSOperations for syncing, observing, and loading files. Visualizing operations could be fun, together with queues. Those are perfect candidates for scopes and spans.

Also, processes can usually be finished or canceled, so that span status could be helpful. The problem was placing tracer calls on operations start and end. Fortunately, objective-c is a highly dynamic ecosystem, and runtime and features like KVO are what we need here.

After playing for some time with NSOperation, I’ve ended up with the following approach: swizzling `addOperation` on a queue and ’start’ on NSOperation, subscribing to `isExecuting` and `isFinished` key paths, and tracing operation start and stop events on appropriate KVO calls. This time result was much closer to what I expected:

This looks more useful and convenient, especially if you have many operations in your app. All you need is a package and tracer code available on the git repo and via CocoaPods.

Conclusion

Custom packages are fantastic for building highly specialized debug tools used only inside your project. It saves time, but in other instances, that might not be the case. Suppose you work on a middle+ size project; the instruments package could perfectly fit there.

On the other hand, developing an instrument can still be a pain in the ass: almost no documentation, no code examples from Apple, need to write XML manually (in 2019), and no one (except Apple guys) can help you. I’m thinking about building something with custom data modelers or using features not covered in session 410.