Archive for the ‘Development’ Category

Building automatic `OptionSet` entries

Last night Zev Eisenberg was asking about option sets. “Do you still have to specify 1 << _n_ manually for OptionSet conformance? There’s no magic?” So I decided to build him some magic. There’s really no reason you should have to manually put in numbers like this:

public struct Traits: OptionSet {
    public typealias RawValue = Int
    public var rawValue = 0
    public init(rawValue: Traits.RawValue) {
        self.rawValue = rawValue
    }
    
    public static let bolded = 1 << 0
    public static let italicized = 1 << 1
    public static let monospaced = 1 << 2
    public static let underlined = 1 << 3
    public static let outlined = 1 << 4
}

This approach requires unnecessary bookkeeping. You have to keep track of the bits you’ve used, especially if you add or insert new options, or reorder the options  you have. It gives unnecessary prominence to the implementation detail values. There should be a more magic way.

So I decided to write him a solution that automatically generated the bit flags and hid those details from the implementation. The result looks like this:

 public static let bolded = generateOption()
 public static let italicized = generateOption()
 public static let monospaced = generateOption()
 public static let underlined = generateOption()
 public static let outlined = generateOption()

You can move things around, add new items, delete old items. It really doesn’t make a difference from a code maintenance point of view (assuming you’re doing this all during development, not after deployment, where you’d want to use availability and deprecations).

To get here, I needed to create a function that would add type-specific options to any type that conforms to OptionSet. I created a global dictionary to store option counts:

private var _optionSetDict: [AnyHashable: Int] = [:]

To be honest, I hate unparented globals like this. However, Swift does not allow adding static stored values in extensions. I couldn’t think of another better way to handle this. I also built a second global to ensure this dictionary would prevent concurrent access, so my counts would be secure:

private var _optionSetAccessQueue = DispatchQueue(
    label: "sadun.OptionSetGeneration", attributes: [.concurrent])

I needed to box my type references since Swift doesn’t allow types to conform to Hashable. They won’t work out of the box with dictionaries. This solution let me use types as keys:

/// Wraps a type to enable it for use as a dictionary key
public struct TypeWrapper<Wrapped>: Hashable {
    private let type: Wrapped.Type
    public init(_ type: Wrapped.Type) {
        self.type = type
    }
    
    /// Conforms to Equatable
    public static func ==(lhs: TypeWrapper, rhs: TypeWrapper) -> Bool {
        return lhs.type == rhs.type
    }
    
    /// Conforms to Hashable
    public var hashValue: Int {
        return ObjectIdentifier(type).hashValue
    }
}

To create a hashable type entry, I just instantiate TypeWrapper with the type.

Sven Weidauer points out I can use ObjectIdentifier directly
Here’s the OptionSet extension that implements the generateOption() magic:

public extension OptionSet where RawValue == Int {
    public static func generateOption() -> Self { 
        let key = ObjectIdentifier(Self.self)
        return _optionSetAccessQueue.sync(flags: [.barrier]) {
            // This should be locked so there's a guarantee that
            // counts are unique for each generated option
            let count = _optionSetDict[key, default: 0]
            _optionSetDict[key] = count + 1
            return .init(rawValue: 1 << count)
        }
    }
}

I’m not sure that I’d ever actually use this approach in code but it was a fun exercise in problem solving. Sven W. adds “Another thing to keep in mind is that statics are initialised the first time they are used. So in different runs of the program the values can differ. Better not persist OptionSets created by this technique.”

You can see the full implementation over at Github. And if you’re curious, you can go back through the change history to see some earlier takes on the problem.

Like it? Hate it? Got suggestions and improvements? (I always mess something up, so there’s a pretty much 100% chance there’s room for improvement.) Drop a note, a tweet, an email, or a comment.

Thanks to Ian Keen, who suggested extending OptionSet directly.

Device-only code: A polite request for help

You may not follow Swift Evolution. A lot of people don’t. It takes a lot of time and attention, and the signal-to-noise while good for lists of its type can be low for people with deadlines, managers, contracts, and real life.

So let me get to the point: do you have code that doesn’t or shouldn’t run in simulators? Maybe you’re building AVFoundation camera code or Metal or for the keychain or whatever? Instead of using tests like:

// Test for a simulator destination
#if (arch(i386) || arch(x86_64)) && (!os(macOS))
    // code suitable for simulator
#else
    // code suitable for device
#endif

wouldn’t it be a lot easier, more robust, and better for cross-platform development to have something simple like this?

// Test for a simulator destination 
#if targetEnvironment(simulator)
    // code suitable for simulator 
#else
    // code suitable for device 
#endif

If you think so, please send a note to kremenek@apple.com and mention your support of SE-0190. You can read the whole proposal at that link. Big thanks go to Graydon Hoare for his implementation.

Time is running short. Review ends on the 24th. This is very much a developer-driven proposal as opposed to a language-design proposal. I’d like to see Swift users have more presence in the SE community.

Thank you.

Styling an optional `Bool`

So  Russell Finn asked me today if I had any thoughts on styling an Optional<Bool>. His use case is something like this: what if you have an optional variable like Optional<NSWindow>, with a property or method on that type, and want to test against the one case where the value is not-nil and the property or method on that value is meaningful?

For example, what if you have a possible window that’s possibly zoomed? How would you style that in the most readable fashion? There are, of course, a lot of ways to express this:

if let zoomed = window?.isZoomed, zoomed { ... }
guard let window = window, window.isZoomed else { ... }
if let window = window {
    if window.isZoomed { ... }
}
guard let window = window else { ... }; 
    if window.isZoomed { ... }

and so forth.

I suppose the thing you should be asking yourself is how long the window is of interest. If the only goal is to dezoom it if it’s zoomed, the if approach makes sense. However, if you plan to perform multiple steps, then you’ll want to unwrap the window and then work directly with the unwrapped version.

There’s nothing particularly terrible about stacking conditions into a compound guard or if statement so long as the conditions are related and tell a single step-by-step story. But if they don’t, I’d recommend breaking them down, as in the second two examples.

If it’s a one and done (thanks Dave and Greg), you can compare directly with a truth value:

if window?.isZoomed == true { ... }

I’m afraid, I’d have to know a bit more about the exact circumstances of use to have a stronger opinion. I hope this helps.

How to `split` without consuming values

Swift’s built-in split function returns an array of subsequences, split from a sequence or collection’s elements. For example, you may have a string and want to split it into words. Here’s a simple approach that consumes whitespace, newline, and punctuation characters:

import Foundation

let string = """
    Lorem ipsum dolor sit amet, consectetur adipiscing elit.
    Praesent in purus a ante semper congue posuere at lacus.
    Nullam faucibus sem vel sem vestibulum, a ullamcorper nunc
    auctor.  
    """

extension Character {
    var firstScalar: UnicodeScalar {
        return self.unicodeScalars.first ??  UnicodeScalar(0)
    }
}

// NOTE: This code demonstrates a simple way to split a collection.
// It's not a great way to actually tokenize a string, lacking both
// localization and in-word punctuation handling such as
// hyphens and single quotes.
let targetSet = CharacterSet.whitespacesAndNewlines
    .union(CharacterSet.punctuationCharacters)
let isWordSeparator: (Character) -> Bool = { c in targetSet.contains(c.firstScalar) }
let words = string.split(whereSeparator: isWordSeparator)

But what happens when you want to split a sequence or collection without consuming values? There’s no built-in Swift solution to turn to.

A few months ago, Soroush Khanlou and I were playing around with this problem, trying to return subsequences around predicate boundaries. When the predicate failed, a new subsequence would begin. We wanted to break sequences  along logical lines, for example, where a value changed or a slope trend updated from increasing to decreasing:

let x = [1, 2, 2, 3, 3, 3, 1]
 .sliced(where: { $0 != $1 })
// [ArraySlice([1]), ArraySlice([2, 2]), ArraySlice([3, 3, 3]), ArraySlice([1])]
let z = [1, 2, 2, 1, 3, 3, 1].sliced(where: >)
// [ArraySlice([1, 2, 2]), ArraySlice([1, 3, 3]), ArraySlice([1])]

We decided to work with arrays and use ArraySlice instances. Each ArraySlice provides a view onto a larger array:

Instead of copying over the elements of a slice to new storage, an `ArraySlice` instance presents a view onto the storage of a larger array. And because `ArraySlice` presents the same interface as `Array`, you can generally perform the same operations on a slice as you could on the original array.

Slices don’t allocate new storage so the splits are efficient. Be careful not to hold onto slices long term. Each slice holds a reference to the array it describes, so you can run into memory management issues if the slice outlives the array it points to.

For safety, reference the start and end of each slice with startIndex and endIndex. These values are often not zero or the array count, the way you’d expect with actual array instances:

let array = ["a", "b", "c", "d", "e"]
let slice = array[1...3]
print(slice.startIndex) // 1
print(slice.endIndex) // 4

Here’s the code I eventually settled on. It extends ArraySlice with a recursive slicing function that ends when it runs out of data or exceeds the maximum partition count. For each slice, it walks the elements until it finds a member element that fails the predicate test, at which point it recurses with a new slice. The ArraySlice code is hidden behind an Array entry point, which wraps the slice functionality away from public view.

Here are some examples of how the code could be used to break apart arrays. They show creating subsequences and run length encoding. You could easily adapt this to more complex data. For example, you might use keypaths to chunk data structures based on time stamps or look for inflection points for rising and falling values.

// Same value
let x = [1, 2, 2, 3, 3, 3, 1]
    .sliced(where: { $0 != $1 })
// [ArraySlice([1]), ArraySlice([2, 2]), ArraySlice([3, 3, 3]), ArraySlice([1])]

// Run-length encoding
let xx = [1, 2, 2, 3, 3, 3, 3, 3, 1, 1, 2, 1, 1]
  .sliced(where: !=)
  .map { ($0.count, $0.first!) }
// [(1, 1), (2, 2), (5, 3), (2, 1), (1, 2), (2, 1)]

// Strings
let y = Array("aaaabbbbcccdef")
  .sliced(where: !=)
  .map({ String($0) })
// ["aaaa", "bbbb", "ccc", "d", "e", "f"]

// Increasing subsequences
let z = [1, 2, 2, 1, 3, 3, 1].sliced(where: >)
// [ArraySlice([1, 2, 2]), ArraySlice([1, 3, 3]), ArraySlice([1])]

// Decreasing subsequences
let w = [1, 2, 2, 1, 3, 3, 1].sliced(where: <)
// [ArraySlice([1]), ArraySlice([2, 2, 1]), ArraySlice([3, 3, 1])]

Got suggestions for improvements? Drop a note in the comments. Thanks!

Matching Regular Expressions

Last week, I wrote about Olivier Halligon’s elegant Set matching solution. Several people asked if this could be extended to pattern matching for other types, which it can. Today, I put together a little proof of concept for performing regular expression matching with strings.

I started off with the same struct plus stand-alone global pseudo-constructor approach I had used with sets. Doing so, let me have a RegexPatternMatcher type and implement a matching function to hide a constructor. After a bit of playing around,  I dropped the global function to use a normal initializer: RegexPatternMatcher("pattern") vs matching("pattern"). The call sites were longer but less garbage floating around.

Renaming the type to Regex provided a more succinct approach : Regex("pattern"). I don’t normally like using overly short type names to balance call sites but I felt that Regex said about all that needed to be spoken here:

I’m not a huge fan of NSRegularExpression. I can’t wait for a native Swift solution. The class is expensive and it relies on what feels like archaic string models that use NSRange. Olivier suggested I balance the expense with NSCache so I created a variation where converted my struct to a class, and used NSString patterns as the cache keys.

This allowed me to build each NSRegularExpression instance once and decouple the matching options from the pattern. I made the class match function ignore defaulted option arguments. Instead, I allow the cache to carry forward any matching policies that previously existed or overwrite those policies if you specify them explicitly.

There’s something wrong with my code because I could not get a cached regex to change its matching behavior even after I updated options. Look at lines 103 through 107 in this version. If you figure out what’s wrong, let me know. It returns true for all six tests, instead of four true then two false.

Anyway, it was an interesting exercise and a good way to start getting back to work as I recuperate.

A Beautifully Elegant way to Set-Match

Challenge: By default, Swift’s switch statement uses equality matching when testing Set instances (for example). So how can you switch that up to use containment instead of equality? For example, say you want to test a set of insets presented as Set<Inset>? Here’s the standard Swift solution for containment:

switch insets {
case let sides where sides.isSuperset(of: [.top, .bottom]) : ...
case let sides where sides.isSuperset(of: .top) : ...
case let sides where sides.isSuperset(of: .bottom) : ...
}

It’s not awful, but couldn’t there be a more beautiful way to allow pattern matching against a set using a more Swifty statement that didn’t abuse its where clause?

Solution: Olivier Halligon came up with the following approach. He built a custom Set containment struct whose pattern matching operator (~=) is customized to apply superset detection:

public struct SetContainmentMatcher<T: Hashable> {
    public let set: Set<T>
    
    public static func ~=(
        lhs: SetContainmentMatcher<T>, 
        rhs: Set<T>) -> Bool {
        return rhs.isSuperset(of: lhs.set)
    }
}

public func containing<T>(_ set: Set<T>) -> SetContainmentMatcher<T> {
    return SetContainmentMatcher(set: set)
}

To use, build your switch using the now-global containing function. The function enables you to test your set against other sets using standard switch pattern matching.

public enum Inset { case top, bottom, left, right }

let sides: Set<Inset> = [.top, .right]
switch sides {
case containing([.top, .bottom]):
    print("contains: top, bottom")
case containing([.top]):
    print("contains: top") // matches here
case containing([.bottom]):
    print("contains: bottom")
default:
    print("default case")
}

Nice, isn’t it? Make sure to test for the largest sets first. If you invert the logic here, testing [.top] before [.top, .bottom], you may end up with unexpected behavior.

More good reading here on Olivier’s blog.

Swift 5 begins: new evolution rules will require implementations for language change review

Ted Kremenek writes about Swift 5 on the Swift Evolution list. Among other changes, SE proposals will now require implementations before Core Team review. Swift 5 will focus on ABI stability and concurrency.

The proposal phase for Swift 4 is now officially over, and the release is now in endgame engineering convergence for an expected release later this year. Swift 4 has turned out to be one of the highest quality, well-rounded releases of Swift, and I am grateful to everyone in the community who made this release come together!

Now it is time to turn our attention to Swift 5. I have just posted updates to the README.md file on the swift-evolution repository, which outlines the core themes and focus areas for Swift 5:  https://github.com/apple/swift-evolution and a more persistent URL (invariant to future README.md changes): https://github.com/apple/swift-evolution/blob/9cc90f33b6659adeaf92355c359e34e6fed73254/README.md

I am not going to repeat all of that information here, but I wanted to highlight a few important things.

ABI Stability

First, ABI stability is the center focus of Swift 5 — and we will pivot much of our prioritization of efforts for Swift 5 around it. With Swift 4, ABI stability was a strong goal. In Swift 5, it is a requirement of the release. Whatever ABI we have at the end of Swift 5 is the ABI that we will have. ABI stability is an important inflection point for the maturity of the language, and it cannot be delayed any longer.

Please note that there is a difference between ABI stability and module stability. If you are not aware of the difference — which is rather important — please read the first few paragraphs of the ABI stability manifesto: https://github.com/apple/swift/blob/master/docs/ABIStabilityManifesto.md

Module stability is a stretch goal for Swift 5, but even without module stability we can still achieve the primary value of ABI stability.

Other focus areas (including laying the groundwork for concurrency)

There are several other areas mentioned for Swift 5 which I won’t repeat here, but there is a general theme of refining and broadening out the core ergonomics of the language and standard library.

One of those that I wanted to highlight is laying the groundwork for concurrency. It is a non-goal of Swift 5 to roll out a full new concurrency model. That is simply too large an effort to do alongside ABI stability. However, it is important that we start making progress on discussing the directions for concurrency and laying some of the groundwork. This may take the form of specific enhancements to the language that get implemented, depending on where the discussions for concurrency lead and how they align with the priorities for delivering ABI stability in Swift 5.

Changes to the language evolution process

Last, I want to highlight important changes to the evolution process for-swift-5:

With Swift 4, the release period was divided up into “stage 1” and “stage 2” for setting guidelines for the kind of evolution proposals that were in scope for the release. This was needed to establish focus — especially after the churn we saw during Swift 3 — on some core themes that were aligned with converging the language towards source & ABI stability. One downside is that “stage 2” opened up discussion for potentially disruptive changes fairly late in the release. Further, some proposals — such as SE-0155 — came in so late that there was little runway to actually implement them for Swift 4, let alone evaluate their impact in practice on real projects. Related, there has been some desire for a while to be able to better evaluate the impact of proposals on real code before they are locked into the release, and the best way to do that is to actually have an implementation that vets out the design in a proposal.

With Swift 5, the focus on ABI stability will predominate priorities for both design and implementation work, but the Core Team did not want that focus to stifle all discussion on potential enhancements to the language that were not fundamentally tied to that primary goal. After reflecting on the evolution process during both the Swift 3 and Swift 4 releases, the Core Team felt that we could strike a balance with not diluting attention from ABI stability while still enabling a broader range of proposals compared to Swift 4 by requiring that all proposals have an implementation before they are officially reviewed by the Core Team. An implementation can come long after an idea has been pitched and after a proposal has been written. However, before a pull request for an evolution proposal will be accepted — and thus an official review scheduled — an implementation must be in hand for a proposal as part of the review. The existence of an implementation does not guarantee that the proposal will be accepted, but it is instrumental in evaluating the quality and impact of the proposal.

There are two key benefits of requiring an implementation:

  1. It encourages a design in a proposal to be more thoroughly fleshed out before the proposal is formally reviewed. The hope is that this will make the review process both more efficient as well as more effective.
  2. An implementation allows the proposal to be evaluated on real world code and not just the examples that are in the proposal.

The Core Team is also sensitive to proposal authors investing time in providing an implementation for a proposal that is not likely to get traction. The Core Team will be regularly reviewing potential proposals, and provide feedback either during the pitch stage or when a proposal is submitted via a pull request on whether or not the proposed change looks within the charter of the release or meets the criteria for a valuable change to the language.

Requiring an implementation naturally raises the bar for proposals. While this is by design, it can possibly have the negative aspect of making some feel the bar is too high for them to participate in the Swift evolution process. As an open source project, both the design and implementation of the language is a highly social endeavor, and we encourage the community to collaborate on both the design and implementation of proposals. Specifically, it is not a requirement that the original author(s) of a proposal be the one who provides an implementation — all that matters is that there is an implementation when a proposal gets reviewed.

Lastly, an important aspect is that unlike Swift 4, the broadening of scope for proposals considered for Swift 5 begins… now! Proposals that fit within the general focus of the release are welcome until March 1, 2018. Proposals will still be considered after that, but the bar will be increasingly high to accept changes for Swift 5.

  • Ted

Dear Erica: How do I simplify `Swift.print`?

Dear Erica:

In other languages (Python mostly comes to mind, but I think ml and probably Haskell as well) you can essentially create a new name for a function by assigning it to a variable and use that instead.  All type information and usage follows with it.

I use it a lot in Python to have a local variable that’s  already bound to some otherwise nested module call. It saves some lookup time and textual typing. (Gotta be explicit since syntactic typing is relevant in Swift.) In trying to work around the problem of “print” calling  NSDocument.print, I tried declaring a val at the top level:

let Print = Swift.print // for print to console

The compiler seemed happy with the definition but when I tried to use it as in:

Print(“in makeWindowControllers”)

I got:

Hitting Fix did nothing but I wasn’t really expecting much (this is Xcode 9.0 beta). I assumed it wasn’t  telling me to literally put <#String#> in as a second parameter but that the type inference was inferring a type of String for  parameter #2  It didn’t really make any sense (why would a second parameter be required for a variadic argument?) If two, why not 3…?

So I tried the following but to no avail:

Print(“in makeWindowControllers","two”)

One can obviously write a full function which works fine:

func Print(_ items:Any...) -> () {
    print(items) 
} 

Print("in makeWindowControllers")

It’s pretty simple but seems unnecessarily heavyweight. I can’t seem to find any way to do a variadic closure. Is this possibly a bug or am I missing something obvious? Is there any reason the simple value declaration shouldn’t work? I know I’ve seen complaints about the lack of an Apply function in Swift. Not sure if this is related. My functional mojo is somewhat lacking…

This is a terrific question, and I’m going to answer it in several parts.

First, the full signature of Swift’s version of print is

public func print<Target>(_ items: Any...,
    separator: String = default, 
    terminator: String = default, 
    to output: inout Target) 
    where Target : TextOutputStream

See all those default items?  They don’t travel nicely to closures.  Second, take careful note of the variadic items, because you can’t pass them along by redirecting them to another function. Your “full function” actually prints an array of the items you pass:

Print("Hello world") // prints ["Hello World"]

The only way around this for 10.12 and earlier is to re-implement print (or a reasonable facsimile). Having tried that this morning, I warn you, it is a doozy. Seriously. You wouldn’t believe how many functions and files go into this one little call.

Fortunately, this name-overlap problem (NSView.print vs Swift.print, etc) is fixed in High Sierra. The 10.13 release notes write:

print() methods in Swift: NSWindow, NSView, NSDocument’s print() instance methods have been renamed to printWindow(), printView(), and printDocument() respectively in Swift 4. This fixes the unexpected experience where adding debug logging to a subclass of one of these instances shows a print panel instead.

You can help the variadics don’t propagate cause (“El viva variadics!“) by filing a radar. Go ahead and dupe, mentioning rdar://problem/12134482.

For the moment, here’s a less capable workaround you can use. It does pretty much what you want with slightly reduced capabilities (and complexity) compared to the built-in Swift.print solution.

public func sprint(
    _ items: Any...,
    separator: String = " ",
    terminator: String = "\n")
{
    var prefix = ""
    for item in items {
        Swift.print(prefix, terminator: "")
        Swift.print(item, terminator: "")
        prefix = separator
    }
    Swift.print(terminator)
}

It’s kind of a production-unfriendly workaround for just avoiding Swift.print, but I hope this helps anyway.

Thanks Dave, Tim, Stephen, Caleb

Beta 3 Playground Workarounds

Adding Resources and Sources folders to Playgrounds

Until they’re fixed, you may have to add them by hand.

  1. Right-click/control-click.
  2. Show package contents
  3. You can add new Resources and Sources files at the top level
  4. Alternatively, navigate down to individual pages (which are finally in Beta 3!) by showing their package contents and add them there

Fortunately under Beta 3 you don’t have to manually add pages, as the new page functionality is finally back.

Creating New Playgrounds

They’re no longer listed in the File > New dialog. Instead choose File > New > Playground (Command-Shift-Option-N) or open “Welcome to Xcode” (Command-Shift-1) and click “Get started with a playground”.

Most of the obvious alternatives (like Command-Control-N, which creates new workspaces) are already taken, but if you don’t mind using the menu for that, I think it’s a nicer key binding for “New Playground”. If you want to mess with this open prefs (Command-comma), type playgrounds into the search field, and edit the key binding for “New > Playground”

Don’t forget that new playground page is “Command-Option-N”.