The surprising awesomeness of Grouped Dictionaries

Yesterday, I was chatting about ways to partition a stream of values. I wanted to collect values into new streams: values that satisfied a predicate, and those that did not. A number of hugely complicated approaches were discussed until Nate Cook brought up a fantastic new Swift 4.0 API. The Dictionary type’s init(grouping:by:) call allows you to convert any sequence to a dictionary by grouping its elements.

Pass the initializer a sequence and a closure, and the initializer creates entries for each value returned by the closure. For a predicate, you end up with two groups: one populated for true predicate values, one for false:

let numbers = 1 ... 20
let predicate: (Int) -> Bool = {  $0 % 2 == 0 }
let grouping = Dictionary(grouping: numbers, by: predicate)
print(grouping)
// [false: [1, 3, 5, 7, 9, 11, 13, 15, 17, 19], 
//  true: [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]]

Once partitioned, you can pull values from each collection (true and false) and operate on the members of that particular group:

// Iterate through the even members of this sequence
for number in (grouping[true, default:[]]) { ... }

This example uses Swift’s new default value in its grouping look-up:

grouping[true, default:[]]

If you haven’t started using this new feature, you should really adopt it into your work flow. It’s wonderful. With this call, a dictionary returns the default value when a key is not found. This avoids forced unwraps (dictionary lookups normally return optionals) and acts as an alias for nil coalescing. In this example, the default call is an alias for (grouping[true] ?? []).

Dictionary grouping provides a solid solution for sequence partitioning by predicates. But you can also do a lot more with this API. Let me give you a bunch of examples that showcase the power of this one little call.

This example creates a dictionary of names grouped by first letter. Swift creates an entry for each unique capitalized letter it finds within the name collection:

let names = ["bob", "Amelia", "joe", "alice", "jane"].map({ $0.capitalized })
let nameDict = Dictionary(grouping: names.filter({ !$0.isEmpty })) {
    $0.prefix(1)
}
print(nameDict)
// ["J": ["Joe", "Jane"], "B": ["Bob"], "A": ["Amelia", "Alice"]]

You could easily expand this example to disregard diacritical marks by stripping them through a StringTransform. (This approach is left as an exercise for the reader. 😀 )

In more realistic text-based grouping, the information you want to group on is often a level or two down within a structure. Swift keypaths make it easy to access the information you need for grouping. This next example constructs a keypath to a contact’s last name, and uses that keypath to provide the partition keys for the dictionary.

/// A person
struct Person {
    let firstName: String
    let lastName: String
}

/// A contact entry
struct Contact: CustomStringConvertible {
    let name: Person
    let address: String
    var description: String { return "\(name.firstName) \(name.lastName) at \(address)" }
}

/// Construct some contacts
let lasts = ["smith", "jones", "simpson", "cheese", "putty"]
let contacts: [Contact] = zip(names, lasts)
    .map({ Person(firstName: $0.0.capitalized, lastName: $0.1.capitalized) })
    .map({ Contact(name: $0, address: "1 Main St" )})

// Establish keypath to the last name field
let lastNameKeypath = \Contact.name.lastName

// Construct the address book based on the first letter
// of the contact's last name, then print the
// address book
let addressBook = Dictionary(grouping: contacts) { 
    $0[keyPath: lastNameKeypath].prefix(1).uppercased() 
}
for (key, value) in addressBook { 
    print(key, value) 
}

// J [Amelia Jones at 1 Main St]
// C [Alice Cheese at 1 Main St]
// P [Jane Putty at 1 Main St]
// S [Bob Smith at 1 Main St, Joe Simpson at 1 Main St]

Grouped dictionaries aren’t limited to strings and numbers. They can be quite helpful when working with interface elements. For example, in a complex user-managed  presentation, you might group button views based on their control state:

// buttonArray is [NSButton]
let buttons = Dictionary(grouping: buttonArray, by: { $0.state })

Although all the examples so far have used sequence data to produce the keys used in grouping, those keys needn’t be pulled directly from the values they categorize. Enumerations make a great Swift choice for categorizing data. This example revisits the even/odd grouping shown in the first example of this write-up but replaces the true/false predicate values with a Parity enumeration.

enum Parity {
    case even, odd
    init(_ value: Int) {
        self = value % 2 == 0 ? .even : .odd
    }
}
let parity = Dictionary(grouping: 0 ..< 10 , by: Parity.init )

You can use this same approach with more extensive enumerations and more complicated data. That said, you can take exactly this example, and simplify it enormously by grouping numbers based on a simple function. For example, you can use the direct results of the modulo operator (which returns 1 and 0)  as the keys to your grouped dictionary:

let parity2 = Dictionary(grouping: 0 ..< 10) { $0 % 2 }

In the end, it’s up to you on how you want to split your sequence, and the keys you want to represent the subsequences derived from that split. Hopefully this post has given you a few ideas to inspire your own partitioning schemes.

If you like my write-ups, please consider buying a book. Thanks to everyone who contributed to the discussion about partitioning sequences, especially Nate Cook, Tim Vermeulen, Kabir Oberai, Daniel Jalkut, Tom Harrington, Soroush Khanlou, Paul Cantrell, and Zachary Drayer.

Working with text output streams

Here’s another great write-up from Tim V. 

By far the most common way to add a custom textual representation to a type is by implementing CustomStringConvertible. Consider this type:

struct Person {
    let name: String
    let age: Int
    let spouse: String?
    let children: [String]
}

A pretty standard way to conform to CustomStringConvertible would be this:

extension Person: CustomStringConvertible {
    var description: String {
        var description = ""
        
        description += "Name: \(name)\n"
        description += "Age: \(age)\n"
        
        if let spouse = spouse {
            description += "Spouse: \(spouse)\n"
        }
        
        if !children.isEmpty {
            description += "Children: \(children.joined(separator: ", "))\n"
        }
        
        return description
    }
}

This code doesn’t look too bad, but it’s a pain to write var description = "" and return description over and over, if this is a pattern you commonly use. It’s also quite easy to forget to add \n to each line.

The relatively unknown standard library protocol TextOutputStreamable solves both of these problems for you. Rather than adding a description computed property, all you have to do is write your properties to a TextOutputStream instance:

extension Person: TextOutputStreamable {
    func write<Target: TextOutputStream>(to target: inout Target) {
        print("Name:", name, to: &target)
        print("Age:", age, to: &target)
        
        if let spouse = spouse {
            print("Spouse:", spouse, to: &target)
        }
        
        if !children.isEmpty {
            print("Children:", children.joined(separator: ", "), to: &target)
        }
    }
}

That’s it! Whenever something that conforms to TextOutputStreamable but not to CustomStringConvertible is turned into a string, the write(to:) method we just implemented is used:

let person = Person(name: "Michael", age: 45, spouse: "Emma", children: ["Charlotte", "Jacob"])
print(person)

>Name: Michael
>Age: 45
>Spouse: Emma
>Children: Charlotte, Jacob

If you enjoyed this write-up, you might be interested in an old post I wrote about using streams to transform data.

Initializing Typed Arrays

Foundation’s URLQueryItem is just a stringly-typed key-value pair. You create one with a name and value:

public init(name: String, value: String?)

Since Swift supports literal initialization, you’d think you could use a dictionary to set up a [URLQueryItem] array, right? Well, yes and no.

You can’t just conform Array where Element == URLQueryItem to ExpressibleByDictionaryLiteral. Array extensions with constraints cannot have inheritance clauses. There are several ways around this limitation.

First (and best), you can just map an initializer across a dictionary literal:

let result = ["key1": "value1", "key2": "value2", "key3": "value3"]
    .map({ URLQueryItem(name: $0.key, value: $0.value) })

Second, you could make URLQueryItem itself dictionary initializable, but that starts to get ugly:

extension URLQueryItem: ExpressibleByDictionaryLiteral {
    public typealias Key = String
    public typealias Value = String
    public init(dictionaryLiteral elements: (String, String)...) {
        guard elements.count == 1
            else { fatalError("URLQueryItem requires single key-value pair") }
        self.init(name: elements.last!.0, value: elements.last!.1)
    }
}

let uq: URLQueryItem = ["key": "value"] // okay
let uqs: [URLQueryItem] = [["key1": "value1"], ["key2": "value2"], ["key3": "value3"]] // bleh

Third, you could use some kind of intermediate type to produce a URL query item array using Swift shortcuts. For example, you can set up a struct that builds the query item array and then pull from there:

struct URLQueryItems: ExpressibleByDictionaryLiteral {
    public typealias Key = String
    public typealias Value = String
    let items: [URLQueryItem]
    public init(dictionaryLiteral elements: (String, String)...) {
        items = elements.map({ URLQueryItem(name: $0.0, value: $0.1) })
    }
}

let uqis: URLQueryItems = ["key1": "value1", "key2": "value2", "key3": "value3"]
uqis.items

But again, it’s really not an improvement on using a mapped dictionary.

In the best of all worlds, which doesn’t exist, you’d be able to do something like this, but I don’t think there’s a way to accomplish this in modern Swift. Solution 2 is about as close as you get.

myRequest.queryItems = ["key1": "value1", "key2": "value2", "key3": "value3"]

Is there something I’m overlooking? If so, let me know. Drop a comment, mail, or tweet. Thanks.

Update:

and

The problem with Swift Playground localization

Starting in Swift Playgrounds 2, you can now use localized strings to guide the narration of your interactive lessons. As the screenshot above demonstrates, you can used localizable markup to provide the most appropriate text for titles, introductory text, and feedback.

However, what you can’t do is localize Swift members. Your French and Chinese consumers must tell Byte to moveForward(), not avancer() or 向前移动().

One of the guiding principles of the Swift language is demonstrated in its embrace of unicode for identifier symbols. This approach accommodates programmers and programming styles from many languages and cultures.

Xcode 9 has introduced major advances in code refactoring. It seems an obvious win to allow that technology to be applied to Swift Playgrounds 2, enabling identifier localization.

That’s because identifiers play such a key role in Swift Playgrounds. Unlike standard development tasks, where it’s unnecessary to create idiomatic APIs like IUContrôleurDeNavigation, the point of Swift Playgrounds is to teach and instruct. It uses small, limited, controlled API exposure, nearly all custom and supporting of the teaching story.

The anthropomorphized Byte character acts as a stand-in for the learner coder. And in doing so, it should communicate with commands that this coder identifies with, turnLeft and moveForward, not incomprehensibleForeignPhrase1 and evenMoreConfusingForeignPhrase2.

I think this is an opportunity waiting to happen, and I can’t imagine it would be all that hard to implement given the expansive identifier suite and the limited API visibility presented in a typical playgroundBook.

What do you think? Is it too much to ask for a localizable.Members.plist?

Simulating a second finger during drag

You can drag and drop in the iOS simulator by clicking and holding an item. The item “pops” and you can then drag it to a destination. Today, an Apple engineer shared a neat way to free up a “second finger” during this process.

Pause and press the control key. This pins an item mid-drag, enabling you to use the Mac cursor as another touch. You can then retrieve your drag by grabbing the paused item and conclude your drop.

State of the Swift PM from the Swift PM PM

From Rick Ballard, Swift Package Manager release manager, a status update with regard to Swift 4. Here’s an overview of the proposals that have recently been incorporated into the system:

We’ve implemented a number of evolution proposal[s] this Spring:

• SE-0152 [ https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md ] introduced a “tools version”, stored in a comment at the top of the Package.swift manifest, which allows you to declare what version of the Swift tools your package requires, and to avoid resolving package dependencies against package versions which require newer tools than those you are using. The tools version is also used to control whether to enable new features such as the revised package manifest API, and to determine which Swift language version is used to interpret the manifest.

• SE-0158 [ https://github.com/apple/swift-evolution/blob/master/proposals/0158-package-manager-manifest-api-redesign.md ] redesigned the Package manifest API, adopting the Swift API design guidelines and cleaning up some problems in our API. Existing packages can continue to use the old Package manifest API until they update their Swift tools version.

• SE-0151 [ https://github.com/apple/swift-evolution/blob/master/proposals/0151-package-manager-swift-language-compatibility-version.md ] introduced a property to control which Swift language version should be used to compile a package’s sources. If this property is not set, the default is determined by the package’s Swift tools version.

• SE-0146 [ https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md ] added explicit control over what products a package vends to clients, and what targets are used to build each product. Packages using the new manifest API must declare any products which they wish their package to vend.

• SE-0175 [ https://github.com/apple/swift-evolution/blob/master/proposals/0175-package-manager-revised-dependency-resolution.md ] removed the conditional “pinning” feature and replaced it with a simpler “resolved package versions” file (“Package.resolved”), along with improvements to SwiftPM’s dependency resolution behaviors.

• SE-0150 [ https://github.com/apple/swift-evolution/blob/master/proposals/0150-package-manager-branch-support.md ] added support for packages which depend on a branch, rather than a tagged version, of other packages. This is especially useful during bringup of a new package, and in-between tagged releases. In order to enforce stability for tagged versions, a tagged package version must only depend on other packages’ tagged versions, not on branches.

• SE-0162 [ https://github.com/apple/swift-evolution/blob/master/proposals/0162-package-manager-custom-target-layouts.md ] added API for controlling where the source files for each target should be found. This allows SwiftPM to support source trees that don’t conform to the standard package layout conventions.

• SE-0149 [ https://github.com/apple/swift-evolution/blob/master/proposals/0149-package-manager-top-of-tree.md ] added support for a “–path” flag to `swift package edit`, allowing users to take full control over an edited dependency and share it between multiple top-level packages.

In addition to these proposals, we’ve made other significant improvements:

• On macOS, package manifest interpretation and the package build are now sandboxed, to help mitigate the effects of maliciously crafted manifests.

• Many error messages and diagnostics have been improved, including information about conflicts during dependency resolution.

• Xcode project generation has been improved, and now allows schemes to reference package targets across regenerations of the project.

• There have been a host of smaller improvements and bugfixes.

All these changes are available in the Xcode 9 beta released today, and in the Swift 4.0 Development snapshots available at https://swift.org/download/#snapshots.

Xcode 9 lays the groundwork for first-class, native support in Xcode for Swift packages with the preview version of its new build system. This build system provides the flexibility and extensibility needed to allow Xcode to support new build models, such as Swift packages. Additionally, considerable work has gone into the SwiftPM library vended by the SwiftPM project, which will support integrating Swift packages into tools like Xcode.

So what’s left in SwiftPM 4? First, we will be implementing SE-0179 [ https://github.com/apple/swift-evolution/blob/master/proposals/0179-swift-run-command.md ], support for a `swift package run` command. Beyond that, we expect to start winding down this release and looking ahead to the next, though we are still open to suggestions or evolution proposals for modest features and fixes.

There are a few features that we originally considered for SwiftPM 4 which are unlikely to be included in this release at this point: performance test support, support for configuration files, support for repositories which contain more than one package, and build settings support. With the possible exception of configuration files, these are likely to be a high priority for the next release. In particular, the core team has done work on a design for build settings which we expect to invite comment and discussion on early in the next release; this is a fairly consequential feature, and we want to make sure to get it right. Since that feature is not landing in SwiftPM 4, we are considering adding some package properties in SwiftPM 4 that will help alleviate some of the biggest pain points here, such as a C++ language version property.

Other features we will likely consider for the next release cycle include support for package resources (such as image assets), license and metadata support, explicit support for handling source control forking, and a generic mechanism for invoking build tools that the package manager doesn’t natively support. Finally, we do anticipate supporting a centralized package index at some point in the future, and we may begin laying the groundwork for that in the upcoming year.

As always, we appreciate the support, feedback, contributions, and adoption we’ve gotten from the package manager community. We’re looking forward to working with you all over the upcoming year to make SwiftPM even better.

Bug reporter complaints

I wasn’t part of the bug reporter preview but having used it today, I’ve got to say that I am not a huge fan of the updated interface. It may be pretty, but for me it’s harder to use. I’d rather see icons and names at the start of the entry screen than scroll through three pages of iconless text (and their subtables) as I try to recognize the phrase that best matches my issue.

It’s funny because this design is a clearly cognitive mismatch between how I used to find the right items, which was just a matter of a single-screen scan, and trying to construct or detect the right verbal phrase that now matches the target issue. The new system is hitting the wrong part of my brain.

If you look at it, it’s laid out very nicely, with big clear areas to enter text and the “drop a screenshot” system is vastly improved over the old system. (Nice job there!) I like the font choices, can live with the sickly purple, and a lot of thought has gone into streamlining issue entry.

That said, the whitespace feels like there’s too much there, the left-to-right range of field width is too hard to scan to review the information you’ve entered, and as I mentioned before, the ‘What are you seeing an issue with?” tables are a little too much for me to process.

If I were to redesign it, I’d:

  • Change the wording to be more business-like: “Classify this issue:”, not “How would you classify this issue?”. “Affected Product or Area”, not “What are you seeing an issue with?”. “Bug or Suggestion”, not “What kind of issue are you reporting” (with the bias being towards ‘bug’, since that’s the primary purpose of the tool.) “Reproducing the issue”, not “What steps can we take to reproduce this issue”, and so forth.
  • In the prompt text, I’d eliminate weasel words. “Step-by-step instructions to reproduce this problem”, not “Please provide a step-by-step set of instructions to reproduce the problem (if possible).”
  • I’d cut back massively on the vertical whitespace, allowing more of the report to be seen at once. There’s just too much floaty gray for my taste.
  • I’d apply one of the “optimal line length” studies, which suggests anywhere between 50 and 75 characters per line for text (which is most of what this tool provides) and 80-160 characters for code entry. (I’d also consider a code-specific “related code” field.) The current width is about 98 characters if I counted correctly. It is very hard to track from the right back to the left while maintaining vertical positioning.
  • I’d explicitly mention Markdown/Common Mark support for bullets and numbering, as well as other markup.
  • I’d change the Suggestion/Bug toggle into either a radio control or a visual toggle with Bug, which should be to the left, pre-selected.
  • I’d massively increase the size of the “Cancel/Save/Submit” buttons and move them from the bottom bar to the main field under attachments.
  • Finally, I’d add a specific call out on completion. Thank the developer for taking time from their busy workday and express appreciation for participating in the radar process. Enroll them either in a rewards-for-radars program or a random giftcard giveaway for the month. It’s a small price to sweeten that “Radar or GTFO” message and it lowers frustration of typing into a big, faceless, low-feedback system.

Those are my thoughts. What are yours?

Apple open sources key file-level transformation Xcode components

Ted Kremenek writes on the swift-dev list:

This afternoon at WWDC we announced a new refactoring feature in Xcode 9 that supports Swift, C, Objective-C, and C++.  We also announced we will be open sourcing the key parts of the engine that support file-level transformations, as well as the compiler pieces for the new index-while-building feature in Xcode.

We will be releasing the sources in stages, likely over the next few weeks:

– For the refactoring support for Swift, there are some cleanups we’d like to do as well as some documentation we’d like to author before we push these sources back.  Argyrios Kyrtzidis and his team from Apple will be handling that effort.

– For the refactoring support for C/C++/Objective-C, these are changes we’d like to work with the LLVM community to upstream to the LLVM project.  These will likely be first staged to the swift-clang repository on GitHub, but that is not their intended final destination.  Duncan Exon Smith and his team from Apple will be handling that effort.

– We’ll also be open sourcing the compiler support for indexing-while-building, which include changes to both Clang and Swift.   Argyrios and his team will be driving that effort.  For the clang changes they will likely be first staged to swift-clang, and then discussed with the LLVM community to upstream them to mainline Clang.

– Finally, we will be open sourcing the remaining pieces of the Swift migrator.  Argyrios and his team will be handling the push back of changes there, and those changes will only be impacting the swift repository.

As usually, we’ll also be pushing back changes to have Swift work with the latest Apple SDKs.  We’re expecting that push back to happen early next week.  When that happens we will temporarily lock commit access to the repositories.  Details about that will be sent out later in a later email.  Until then, the downloadable toolchains from Swift.org will continue to work with Xcode 8.3.2.  After we do the push back the downloadable toolchains will be moved to be baselined on the Xcode 9.0 betas.  This shift is necessary as changes to the overlays depend on the latest SDKs.

Xcodes and Kittens: These are a few of my favorite things

Fully redesigned source editor: You can now increase and decrease the source editor font using Command-plus and Command-minus. I have been waiting for this for years. Other highlights include an improved find and replace system, better scrolling, and integrated markdown support.

Better refactoring. Global renaming works across Swift, ObjC, IB, and other file types.  Apple says you can view changes in a unified presentation. The naming strategies for properties, getters, setters, and synthesized ivars are all coordinated.

Better boilerplate. “Fix-it automatically fills in missing cases in switch statements, and mandatory methods for protocol conformance with one click.”

Better Assets including Named Colors. Support for named colors, wide-gamut icons, HEIF images (the ones they discussed during the keynote this morning), and of course, the larger iOS Marketing images.  I suspect “Added option to preserve image vector data for matching Dynamic Type scaling” won’t be as exciting as I’d hope, but I’m looking forward to finding out more.

Github Integration. “GitHub account integration for easy browsing and one-click creation of a project and the associated GitHub repository.”

First class playgrounds. New playground templates designed to run   (depending on the target) on macOS, in the simulator, and on iOS Swift Playgrounds. If you’re looking for them, choose File > New > Playground. They’re not in “File > New” anymore.

What’s New: Things that caught my eye

I love the idea of ARKit, although I wonder about the practicalities of how people will interact with their iDevice while holding it up to look through it. You build a basic AR experience with SceneKit or SpriteKit, and then configure real world anchors based on flat surface destinations. ARKit can pull ambient light estimates for the captured scene, provide overviews of the visualizing camera, and allow you to “hit test” against modeled sufaces.

CoreML looks really really cool. It leverages training data sets to build models that allow you to predict outcomes for new data as it arrives. Apple has introductions to obtaining CoreML models, integrating them into your apps, and importing third-party machine learning to Apple’s framework. The framework is pretty sparse, so it looks like most of the magic will happen behind the scenes.

It was going to happen eventually, and DeviceCheck allows you to access per-device (developer-specific) data that lets you track which devices have previously interacted with your servers and software.

The new Vision framework offers high-performance image analysis and computer vision tech for face identification, feature detection, and scenery classification. Looks a lot like OpenCV-style stuff at first glance.

You’ll be able to create app extensions that help filter unwanted messages using the new IdentityLookup framework, and provide workarounds that respond to unwanted received messages.

The ColorSync framework doesn’t have much there to play with, but its few constants intrigue me.

The FileProvider (and associated FileProviderUI) frameworks are both major components for accessing documents across applications. We’ve seen a bit of this before, but the new fully fleshed out APIs are fascinating.

Finally there’s CoreNFC, which detects and reads NFC tags and their NDEF data. This framework is limited to the iPhone 7 and 7 plus.

Don’t forget to check out the “diff overview” here.