The problem with macOS and Drawing

Trying to write cross-platform code for drawing routines is ridiculously frustrating. It’s never just a matter of creating cross-platform type aliases. UIColor and NSColor are cousins, not brothers. UIBezierPath and NSBezierPath are slightly more distantly related, especially when it comes to adding curves and retrieving an underlying cgPath.

Although a lot of tech debuts on Cocoa, it often gets refined in Cocoa Touch. That refinement doesn’t prioritize cross-platform development, so the “second time, better design” features that leap into iOS land are hard to support with a single code-base. (And yes, I’m looking at you layout constraints with their different dictionary types for metrics.)

Starting last year, UIKit drawing really improved. It already had several advantages over Cocoa drawing, but last year’s update pushed the APIs to a new level.

NSImage has its  init(size:flipped:drawingHandler:) initializer (introduced in 10.8). in iOS 10, UIImage adopted and refined that closure-based drawing approach to UIGraphicsImageRendererUIGraphicsImageRendererFormat, etc. A renderer can return an image or PNG/JPEG data. Very handy, very convenient, wide-color aware, nifty. And it phases out a lot of home-built solutions that devs had been using to get exactly those results.

The new features are not fully refined. For example, there isn’t a single doc comment across the entire feature suite. Which, you know, sigh. But the architecture is there if not perfect or final, and it’s several steps beyond the corresponding init on macOS.

With this new tech in place, I’m starting to wonder about some older APIs like Cocoa Touch’s drawing stack, which lets you push and pop contexts for immediate drawing. It’s handy for using UIKit calls to draw into custom Core Graphics bitmap contexts. Is that still going to be around and used after WWDC this year? Or next year? Or will it be subsumed into UIGraphicsImageRenderer? It looks ripe for redesign and replacement.

I also don’t think UIGraphicsBegin/EndImageContext is long for the world. It was first introduced in iOS 2. It may be deprecated in iOS 11. The new APIs feel more appropriate to the design and philosophy of iOS, and these calls always stood out as a very odd approach.

But let me get back to the issues of writing “Swift Drawing”. When it comes to book, platform dichotomy hits hard. To get a sense of how much effort it would take to write about drawing for both platforms, I worked on my blend mode appendix sample code this weekend.

This is extremely simple code. It just iterates through  the standard Core Graphics blend modes to create stock samples. The appendix describes each mode We’re talking about maybe a page of code. Getting it to Cocoa? Not so easy.

I put together a very rough skeleton granting Cocoa the basic abilities of Cocoa Touch. I’m not very happy with this code and I’m not sure I’m going to keep pushing on this code or developing its capabilities, which is why it’s in a gist and not a repo.

I suspect we may see new image APIs — both Cocoa and Cocoa Touch — this WWDC. I don’t want to compete with Apple APIs and I also don’t want to have to write two completely distinct drawing books: one for iOS and one for macOS.

So I’m still struggling because while most basic calls for CoreGraphics are interchangeable across platforms, the Cocoa and Cocoa Touch layers are not. If I go with samples that support both platforms, I have to pick one of the following approaches:

  • create an artificial UIKit for Cocoa (that is a disconnect for macOS readers),
  • create an unrealistic platform-independent backbone to unify Cocoa and Cocoa Touch, which means I can’t talk to Cocoa Touch readers in their native language (that is objectively bad), or
  • stick with Cocoa Touch like I did in the original book (simplest but excludes macOS readers).

After this weekend’s experimenting, I’m leaning towards keeping things mostly iOS-centric. I may mix in a bit of “fake UIKit”.

I know I don’t want to go the route of “Color”/”Font”/”BezierPath”/”LayoutConstraint” cross platform types that I use in my own development work. It’s fine to use pseudo-types and typealiases internally but it’s not a way to introduce native tech in a book. Books should teach to the platform with the least distance between the reader and the APIs. No dependencies, few workarounds, minimal obfuscation.

I could build a small set of “fake UIKit” calls for macOS samples. This would allow me to focus discussions on the true cross-platform features like gradients, paths, colors, and bitmaps, without having to write two entire books at once, but I’m not sure it’s worth the effort or the hardship on macOS readers.

The costs look like this:

  • Full macOS support would double my work.
  • Adding compatibility discussions and partial macOS code would add about 60% or more work.
  • A minimal “fake UIKit” and a few references in-book to this approach, I could probably squeak by with about 25-35%.
  • Based on initial feedback, macOS sales would be about 10% at most of my readers.

Bottom line: macOS is a big old monkey wrench for this particular project. After investing a weekend, I’m not sure I want to go there but I’m interested in hearing your feedback after looking at my samples and prototype gists.

11 Comments

  • Have you worked with CoreImage much? Are the challenges of doing cross platform as significant?

    • The shallow answer is CoreImage challenges are mostly looking up what filters are supported on what platforms and what OS release for plug and play.

      I’ve encountered problems when deploying and not being exact about which version of OS X I was supporting. Turns out Apple tested one release earlier than the filter existed because I read the docs wrong, and filters are very stringity and…I’m very glad Apple taught me that lesson and not the App Store customer base.

      The deeper answer is that CI has a lot of performance issues and startup costs.

  • I imagine you wrote this at least somewhat in response to my comment on your previous post. If so, thank you for giving it such a thorough response. I’m sad that you’re not likely to support any macOS drawing, but I get it that it just wouldn’t make sense to support macOS. Thank you again for this, and may your book sell millions of copies so 10% starts to look worthwhile 🙂

  • Well, you’ve written multiple books. I don’t, and can only assume it takes a lot of discipline, as well as figuring out how to make money with a book. If I was you, I’d make it easy for myself. If you’d _love_ to do do full macOS support, and it makes it actually easier to get the book out, do that. Else treat it as one of your “stretch goals”, in Kickstarter parliance.

  • I suggest writing the iOS focused drawing book, and adding an appendix for macOS drawing. In that appendix explain why you didn’t write the book as a cross platform title, and point out many of the drawing things a macOS developer will need to do differently from an iOS developer.

    The book would still be valuable to macOS developers, because at the higher levels the understanding needed to draw in code in iOS is transferable to macOS. The details are what changes, and the appendix can mention many of those details and show where to find documentation of the rest of the details.

    This post is a very good start on that appendix.

  • I think trying to merge UIKit and AppKit drawing concepts via typealiases/extensions is destined to end in a real mess. The concepts are almost all the same, but the details are enough different that you can’t really easily hide them in a platform-independent way. More and more little platform differences end up bubbling up to the actual app drawing code. Maybe Apple makes this better eventually, but they haven’t managed to yet.

    If you want to do platform independent drawing, the way to go is to work entirely at CoreGraphics/CoreAnimation. Code for drawing and animating layer trees can be 100% shared, and then only the event interaction needs to be written separately on each (which _should_ be different because mice aren’t fingers).

    As far as book writing goes, I suspect that the previous recommendation by Eric (i.e. iOS focused with macOS appendix) is probably the best way for a book that has to present the concepts (what is a path, what is a colorspace, etc) as well as the code.

  • I’m mainly interested in the MacOS, but I would be fine with a book that focussed on iOS and had a chapter or two describing the differences under MacOS (with perhaps some footnotes referencing those chapter dotted about in the main text “this API is called foo under MacOS – see p192”). I would get a lot of value from the description of the concepts (which I assume are broadly similar) – I’m quite happy to dig around on the details of API differences myself.

    Please write this book! I’d buy it even if it was purely iOS with no mention of macOS.

  • if your effort doubles I’d still buy it at double the price 🙂

  • I’d also gladly pay double the price. Please write this book.

  • I will pay double the price as well. A friend of mine rewrote a lot of QuickDraw routines because he was sick of CoreGraphics the moment it came out. I also wondered why Apple had to stray so far from the elegant work of Jim Atkinson. So I helped him. We did this because we liked THINK Pascal and he made an IDE for FPC and have this library in Pascal. I am tired of writing new graphics library code. I was thrilled when I found out you were going to do all the work for me. I am now depressed but COMPLETELY understand your viewpoint. :). I’ve see your Coe snippets in this area and know you know about how to do this. New Graphics needs to go into something likeFoundation+ and Apple needs to spend the money to do it IMO.

    Sh_t, I’ll pay triple. 🙂

  • For a cross-platform CG wrapper, you might want to have a look at MPWDrawingContext, https://github.com/mpw/MPWDrawingContext