Archive for the ‘Swift’ Category

Swift Stories: Please share yours

I’m looking for people who have intentionally avoided transitioning to Swift or who get frustrated with Swift due to changes in the language or who have fought for Swift adoption at their place of work. If any of these scenarios apply to you, please send me an email at erica at ericasadun dot com.

Please let me know if I can use your name or not and what your personal story is in terms of Swift adoption (or lack thereof). It would really help if you let me know the big picture reasons motivating your choices.

Thank you in advance.

p.s. For those confused by this post, I’m doing a talk about Swift adoption and participating in the Swift Evolution process: “The future of Swift belongs to those who show up”.

Evolve Swift: Part 1 – Building your Tools

Equipment

If you’ve made the slightly rash but equally rewarding decision to dive into developing the Swift language then make sure you have access to a fast computer with good, fast storage. What you don’t want to do is be running an old Mac mini using an old external hard drive because Apple hasn’t updated the mini line in a few forevers.

If your “About this Mac” panel looks like this, seriously question why you’re trying to do this in the first place. For the sake of uncomplicated narrative, I’m going to assume that just about everyone out there in the real world has a better Mac with better memory and primary disk than I do:

For those interested in the naming from these screen shots: Esopus Spitzenburg., Kiku, and Broxwood Firewhelp. I have others but I think those are the names you might see in screen shots and text examples. Hopefully this will be the first of several posts about the topic, in honor of Paul Cantrell.

Embrace the Wait

Before you can change the compiler, you must get the compiler downloaded, compiled, and working in its most recent form. For me, this is multi-day event. You probably will have slightly fewer roadblocks. Still, keep your day open and focus on other work as your Mac may or may not wind tunnel its way through the process.

Even after you’ve got your toolchain under control and build, you’ll continue to have many many waits in your development process as you’ll want to keep updating your repo to match the current state of Apple’s master and its supporting tools. Each time you do this, you can expect a morning of recompiling before you can start working on anything significant.

During this time, you cannot reasonably explore, test, and build unless you do so on a separate machine or for distinct installs. It’s a big pain in the ass but it’s part and parcel of developing the language.

If you plan to do this casually, and a week  has passed between the last pull from upstream/master and now, don’t expect to be productive until the afternoon at the earliest.

Although Swift’s toolset provides a quick iteration tool (called “Ninja”), that adds fast incremental builds, getting up to date and staying up to date means a large time commitment.

Further, don’t be shocked when your compiler or tools simply fail to build. They do that sometimes. It’s utterly frustrating to be stuck with a broken build for whatever reason. Incorporating workarounds into your toolset is a big part of swifting.

If you can’t embrace the wait, you don’t want to be developing swiftlang.

Selecting Xcode

Ensure that your default Xcode is set correctly. Use xcode-select -p to check which version is selected. If it isn’t your main app, use the utility to establish the right one. I use the production (not beta) version of Xcode:

sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

Package Managers

Make sure you have a good package manager installed. I use Homebrew. I know there was some kind of controversy about that a while ago but I can’t remember why and it does the job for me.

Update your package manager to the most recent version:

brew update

If you run into any issues, you can do a full remove and re-install. These calls require bash and will misfire in csh/tcsh:

Uninstall Brew (removes all existing packages):

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"

Install:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Get cmake and ninja

Grab a copy of cmake and ninja if you do not already have them installed on your system. CMake provides a cross-platform build/test/package solution that generates appropriate Make files, specialized to your environment. Ninja offers a speedy solution for incremental builds. It’s really fast, so once you’ve built out your libraries, you can go in and ninja swift for a really fast update when you’re doing edit/build/test cycles.

brew install cmake ninja

Go ahead and install sphinx now too. Sphinx “makes it easy to create intelligent and beautiful documentation” :

brew install sphinx

Notifications

Because of the long, long nature of setup and update process, consider some tools to help keep you on top of tasks, so you know when they finish and can move on to your next steps.

I use a command-line utility called noti to announce when each stage of downloading and compilation completes. Noti is an open source tools that triggers feedback as a command line process finishes.

You can have the utility speak, create a macOS system notification, and more. It’s helpful for when you’re running long tasks in background windows. Thanks goes to Scott Robbins for his recommendation.

To install noti, follow the directions in the repo’s Readme file. If you elect to curl a copy to your system, use bash, not csh (or in my case tcsh), for copy/pasting to your command line. The utility runs fine in tcsh but its escaping assumes bash is running.

Once installed, just prefix any command line command with noti followed by flags. I prefer using noti -b -s, which gives me both a banner alert and spoken text.

% noti --help
Usage:
  noti [flags] [utility [args...]]

Examples:
noti tar -cjf music.tar.bz2 Music/

Flags:
  -t, --title string     Set notification title. Default is utility name.
  -m, --message string   Set notification message. Default is "Done!".
  -b, --banner           Trigger a banner notification. This is enabled by default.
  -s, --speech           Trigger a speech notification.
  -c, --bearychat        Trigger a BearyChat notification.
  -i, --hipchat          Trigger a HipChat notification.
  -p, --pushbullet       Trigger a Pushbullet notification.
  -o, --pushover         Trigger a Pushover notification.
  -u, --pushsafer        Trigger a Pushsafer notification.
  -l, --simplepush       Trigger a Simplepush notification.
  -k, --slack            Trigger a Slack notification.
  -w, --pwatch int       Monitor a process by PID and trigger a notification when the pid disappears. (default -1)
  -f, --file string      Path to noti.yaml configuration file.
      --verbose          Enable verbose mode.
  -v, --version          Print noti version and exit.
  -h, --help             Print noti help and exit.

Fork Swift and Clone

Starting on Github, log into your account and then fork Apple’s Swift repo. Navigate to github.com/apple/swift, and click fork:

My fork uses my erica github account. Adjust the following directions accordingly so they correspond to your fork.

Hop over to a nice fast drive, and create a development directory. Enter the directory (cd swiftdev, for example) and clone the Swift repo:

git clone https://github.com/erica/swift.git

If you want to get a notification and time how long this took, your command may look like this:

noti -b -s time git clone https://github.com/erica/swift.git

I’ll skip the noti and time references from here on so they don’t clutter up these instructions. I like how time-ing gives you a sense of how long each task takes.

Install Supporting Utilities

Grab copies of the supporting utilities using the built-in update-checkout utility. There’s quite a few supporting folders that need to be built, including llvm, clang, and more. If you’re following these instructions, make sure you’re in the same top level directory as your swift install and not in the swift repo. Otherwise, adjust the path for the update-checkout to drop the initial swift:

% pwd
/Volumes/Kiku/Development
% ls
swift
% ./swift/utils/update-checkout --clone
Skipping clone of 'swift', directory already exists
Running ``obtain_additional_swift_sources`` with up to 8 processes.
Cloning 'compiler-rt'
Cloning 'llvm'
Cloning 'swift-xcode-playground-support'
Cloning 'swift-corelibs-foundation'
Cloning 'clang'
Cloning 'llbuild'
Cloning 'cmark'
Cloning 'lldb'

Now wait for everything to clone. There’s another way to clone using ssh, but I’ve never done it. I just know that it exists and is documented on Apple’s Swift repo Readme file.

Build

You should now be ready to perform a release build. I’m differing here from what’s recommended on Apple’s page because I got help early on from people who didn’t use that method. I’m sticking with my notes because they’re tried and trusted, not because I know what I’m doing.

% cd swift
% noti -s time utils/build-script -R

Building Swift (and, this first time, all its supporting utilities) is not…swift. So go do something else for a significantly long time. Unlike your initial Swift build and when you perform updates by pulling from Apple’s upstream master, incremental changes can be relatively quick. Getting things done the first time and after each update is long and tedious.

Test

If you’ve built things properly, you’ll find a newly build swift compiler waiting for you in the build folder. This directory is located at the top level of your development folder:

% ls

./                       lldb/
../                      llvm/
.DS_Store                ninja/
Hold/                    swift/ [1]
Notes.txt                swift-corelibs-foundation/
build/ [2]               swift-corelibs-libdispatch/
clang/                   swift-corelibs-xctest/
cmark/                   swift-integration-tests/
compiler-rt/             swift-xcode-playground-support/
llbuild/                 swiftpm/

Your cloned swift repo [1] sits along the build folder [2]. Navigate to build > Ninja-ReleaseAssert > swift-macosx-x86_64 > bin to find the swift executable. Check the modification date and run it with the -version flag to confirm it is the right build:

% ls -l build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift 
-rwxr-xr-x  1 ericasadun  staff  96294356 Apr 30 09:23 build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift*

% build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift -version
Swift version 4.2-dev (LLVM bb2a4e8df7, Clang 56384a48b9, Swift 5f9bf115b3)
Target: x86_64-apple-darwin17.5.0

Go ahead and use it to compile a file. Use the path to this build to ensure you don’t accidentally use the default compiler:

% cd ~/Desktop/
% lns ~/github/apple/swift/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift swift
Created symbolic link at /Users/ericasadun/Desktop/swift
% echo 'print("Hello World")' > test.swift
% ./swift test.swift
Hello World

Conclusion

Getting your initial toolset built and running is the first step on the path to Swift evolution. Next up: ninja builds, creating branches, stashing differences, and managing test builds.

Playing with Strides

Some stride-related discussions took place on the forums today. I was inspired to play with strides as I took a break between my kids’ appointments. If any of this resonates with you as a potential user for it, let me know.

Index Strides

The following extension allows you to stride over a collection’s indices.

extension BidirectionalCollection where Index: Strideable {
    /// Returns the sequence of collection elements (`self[start]`,
    /// `self[start + stride]`, `self[start + 2 * stride]` ... *last*
    /// where *last* is the last element in the collection whose index
    /// is less than or equal to `end`.
    ///
    /// - Note: There is no guarantee that the element at index `end`
    ///   is part of the sequence
    ///
    /// - Parameter start: The starting index
    /// - Parameter end: The upper index bound
    /// - Parameter distance: The stride for each successive index
    /// - Returns: A lazy sequence of collection members
    public func stride(from start: Index, through end: Index, by distance: Index.Stride) -> LazyMapSequence<StrideThrough<Index>, Element> {
        return Swift.stride(from: start, through: end, by: distance)
            .lazy.map({ self[$0] })
    }
    
    /// Returns the sequence of collection elements (`self[start]`,
    /// `self[start + stride]`, `self[start + 2 * stride]` ... *last*
    /// where *last* is the last element in the collection whose index
    /// is strictly less than `end`.
    ///
    /// - Parameter start: The starting index
    /// - Parameter end: The upper index bound
    /// - Parameter distance: The stride for each successive index
    /// - Returns: A lazy sequence of collection members
    public func stride(from start: Index, to end: Index, by distance: Index.Stride) -> LazyMapSequence<StrideTo<Index>, Element> {
        return Swift.stride(from: start, to: end, by: distance)
            .lazy.map({ self[$0] })
    }
}

Here’s an example that uses this approach to form a sequence that strides through the collection:

let myArray = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"]
for value in myArray.stride(from: 0, through: myArray.count, by: 3) {
    print(value) // a, d, g, j
}

Range Operators

The next snippet uses the division operator to create its stride. This was pitched by “Tony Y” in a forum thread.

extension CountableClosedRange {
    static func / (range: CountableClosedRange, divisor: Bound.Stride) -> StrideThrough<Bound> {
        let distance = range.distance(from: range.startIndex, to: range.endIndex) / divisor // deprecated
        return Swift.stride(from: range.lowerBound, through: range.upperBound, by: distance)
    }
}

In this example, you just “divide” the range:

for idx in (0 ... 100) / 5 {
    print(idx) // 0, 20, 40, 60, 80, 100
    // yes that's 6 not 5, but isn't that what 
    // you want? 5 subranges between each index?
}

Because I wrote this fairly quickly, I used the deprecated range.distance function as straight index arithmetic seemed to error:

error: binary operator '-' cannot be applied to two 'ClosedRangeIndex' operands
        let distance = (range.endIndex - range.startIndex) / divisor

As usual, I probably messed up somewhere (or several somewheres) or you might see ways I could improve the code. Let me know!

Update: After some thought, I decided I preferred a sequence of subranges (which should probably be a type and not like I do it here):

extension CountableClosedRange {
    static func / (range: CountableClosedRange, divisor: Bound.Stride) -> UnfoldSequence<CountableClosedRange<Bound>, (CountableClosedRange<Bound>?, Bool)> {
        let distance = range.distance(from: range.startIndex, to: range.endIndex) / divisor
        let rangeStride = Swift.stride(from: range.lowerBound, through: range.upperBound, by: distance)
        var indices = Array(rangeStride)
        indices.append(range.upperBound)
        indices.reverse()
        guard var current = indices.popLast(), var next = indices.popLast() else {
            fatalError("Guaranteed count failed in index population")
        }
        var done = false
        return sequence(first: current ... next.advanced(by: -1), next: { _ in
            guard !done else { return nil }
            (current, next) = (next, indices.popLast()!)
            if current == next {
                done = true
                return next ... next
            } else if next == range.upperBound && indices.isEmpty  {
                done = true
                return current ... next
            } else {
                return current ... next.advanced(by: -1)
            }
        })
    }
}

for idx in (0 ... 99) / 5 {
    print(idx) // 0...19, 20...39, 40...59, 60...79, 80...99
}

for idx in (0 ... 100) / 5 {
    print(idx) // 0...19, 20...39, 40...59, 60...79, 80...99, 100...100
}

This isn’t a debugged solution. For example, trying to divide 1 … 1 by 1 or 1 … 2 by 1 will both fail.

Forcing Compiler Errors in Swift

Thanks to SE-0196, Swift 4.2 introduces #warning() and #error() compiler directives. These will allow you to incorporate diagnostic messages and emit errors during compilation. Here are some examples from the proposal, which has already been accepted and implemented:

#warning("this is incomplete")

#if MY_BUILD_CONFIG && MY_OTHER_BUILD_CONFIG
  #error("MY_BUILD_CONFIG and MY_OTHER_BUILD_CONFIG cannot both be set")
#endif

The #error example uses conditional compilation flags (set with a -D option) to check whether conflicting configurations have been established for the build.

I’ve already written extensively about my dislike for screaming snake case (THINGS_LIKE_THIS) in Swift. Inevitably, it seems, devs use screaming snake case for their conditional compilation flags, whether MY_BUILD_CONFIGMY_OTHER_BUILD_CONFIG, or DEBUG. Although an industry standard, it feels like a clash with Swift’s aesthetics.

I’ve also written about my proposal for detecting debug conditions without having to supply an explicit DEBUG condition flag, so I’ll also leave that topic to the side for now. You can click on the link for more.

Back to topic, Swift’s newly adopted #error and #warning directives represents a big step up from current practices, which often rely on run-time rather than compile-time feedback.

#if !DEBUG
fatalError("This code is incomplete. Please fix before release.")
#endif

The unindented style in this snippet is now Swift default, avoiding minor doom pyramids for conditional compilation blocks. Even unindented, surrounding code with those conditional blocks is vertically expansive and subjectively ugly. To counter this, some coders have come up with in-line ways to force compilation (not run-time) errors with minimal condition blocks and a more succinct point-of-use approach.

Here’s an example I discovered from John Estropia. (He, in turn, cribbed it from one of  his co-workers.) He uses conditional compilation to set a TODO or FIXME (or whatever) typealias then uses it in-line at points where a debug build should compile and release builds should error:

#if DEBUG 
internal typealias TODO<T> = T
#endif

print("Remove this later") as TODO

It’s clever. Scoping the TODO typealias to debug builds allows lines annotated with as TODO to throw errors during release builds. This ensures compile-time feedback at all points where a TODO cast is performed:

error: ManyWays.playground:5:31: error: use of undeclared type 'TODO'
print("Remove this later") as TODO

It’s not beautiful but it’s effective. It carries information about the call site location and the message you want to emit. If I were applying this hack, I’d probably build an actual todo function rather than using the casting-gotcha. In the following example, I went with an upper camel case name to make the call look more directive-y and less like a standard global function. However, I drew the line at snake case:

#if DEBUG
internal enum IssueLevel {
    case
    mildImportance,
    moderateImportance,
    highImportance,
    criticalImportance
}

internal func ToDo(_ level: IssueLevel, _ string: String) {}
#endif

// The point of use offers a compilation error,
// a note, and a priority level
ToDo(.highImportance, "Remove this later")

// error: ManyWays.playground:13:1: error: use of unresolved identifier 'ToDo'
// ToDo(.highImportance, "Remove this later")

The nicest bit is that toggling from debug compilation to release is completely automatic and centralized to a single #if check.  It’s a fascinating approach if adopted consistently and ensures that all compilation message notes like this must be resolved and removed before release.

Right now, Swift does not support a #message directive, which performs a similar tasks. As many shops treat warnings as errors, they cannot establish a nuanced distinction between the two. If #warning were a thing, you could use #message to issue exactly this kind of “fix me” feedback. A further refinement, #messageOrDie (or something like that, because naming is hard)  could message for debug builds and error for release, going by whether assert statements would or would not fire.

Dave DeLong offers another approach for structural project semantics. His introduces a Fatal type to provide runtime cues for common development outcomes including notImplemented(), unreachable(), and mustOverride(). Nothing says “you need to remember to implement this” better than a spectacular runtime crash that explains itself with full position and function context. Another cue, brilliantly named silenceXcode() allows you to add methods that you never intend to implement and which should error if ever called.

There’s still space in Swift for expanding this metadevelopment support. I wouldn’t mind seeing both approaches added to the language: one for compile time (like #messageOrDie) and another for run time (like Fatal‘s namespaced static error members).

What do you think of these? And what parts of the metadevelopment process (like macros) are still MIA for you in Swift? Let me know. I’m curious to hear what else could be better established to support your development.

 

Writing Swift: Adventures in Compiler Mods

Ever since Swift adopted the “implement first, propose second” rule, some contributors to the Swift Evolution process have felt sidelined and dropped out of the community. It’s frustrating having ideas for positive language changes and not being able to implement those changes.

Despite expertise in Swift and Objective-C (not to mention any number of other languages), they like me may not be proficient in C++ and Python, the core tools of Swift language implementation. To date, my code contributions to Swift have been extremely low level.

I think I fixed a comment, added a string, and maybe one or two other tiny tweaks. (I did work on the Changelog file a while back but that is written in Common Mark, and does not involve programming in the slightest.)

I’ve wanted to be able to build what I can dream. And I’ve slowly been diving into the compiler in recent months to see what it takes to build something new. With quite a lot of hand holding from John Holdsworth, I implemented a couple of build directives to test whether (1) asserts can fire and (2) a build is optimized.

What is Debug?

Answering “what does ‘debug’ mean?” was a harder question than I initially thought. Coming as I do from Xcode, where ‘debug’ means a scheme I select from within the IDE, it took a bit of thinking and advice to think about ‘debug’ from a platform independent viewpoint. After going back and forth on the Swift Evolution email list and later the forums, the consensus centered on the two tests I mentioned above: assertions and optimization.

For many projects, a typical debug build is unoptimized where asserts can fire. As projects move into the beta process, that mindset changes. Many in-house and beta builds meant for wider use need optimization.

The state of the art uses the custom conditional compilation flags set with -D . This approach decouples the meaning of ‘debug’ (or for most developers 'DEBUG') from anything in-language that can be decoupled from build settings and persist through source code. Assert configurations have their own flag, -assert-config <#value#>.

Introducing these two tests lets you align your not-for-release code to assert states and/or optimization states:

#if !configuration(optimized)
    // code for un-optimized builds only
#endif

#if configuration(assertsWillFire)
    // code where assertions can fire
#endif

The proof-of-concept implementation coupled with a proposal means I may be able to submit a more substantial and meaningful contribution to the language.

Going Solo

Pushing forward, I wanted to test myself and check whether I could make changes on my own, even if that solo journey was quite small. I started by attempting to build the #exit directive that was being discussed on the forums. This turned out to be a little more complicated than I was ready for.

Among other challenges, #exit may or may not be embedded in an #if build configuration test. Using #exit should allow all material up until that point to be compiled into the build while gracefully excluding all material after. I didn’t know how to check whether the directive was embedded in a condition and how to properly complete the condition (#endif) while discarding remaining text. It was, at my stage of the journey, a step too far.

I put my first attempt to the side and tried something else. I tried to push a scope using with(value) { }, so the material within the scope was native to value. That too proved too difficult without assistance although I am beginning to understand how Swift creates and manages its scope. It was a programming failure but a learning success.

Two projects abandoned, I knew I had to pick something very easy to work with. Although I would have loved to have picked up and run with Dave DeLong’s context pitch (which is discussed here) , I recognized that I needed to bite off something smaller first. So I decided to add a #dogcow token that produces the string value `”????????”` in source. How difficult could that be, right?

About five hours and edits to twenty-one files later, I had it working. Kind of. Because I ran into one of the many frequent headdesk situations that plague Swift compiler development. I had focused on my edits to the most recent repo without rebuilding the supporting tool suite.

Ninja Builds

A ninja build is a quick way to build just the compiler component. But at some point you can’t ninja your way into an entire working toolchain. I couldn’t test my changes until I rebuilt everything, a process that can take many many hours on my Mac:

% ./swift ~/Desktop/test.swift
:0: error: module file was created by an older version of the compiler; rebuild 'Swift' and try again: /Volumes/MusicAndData/BookWriting/LiveGithub/Apple/appleswift/build/Ninja-ReleaseAssert/swift-macosx-x86_64/lib/swift/macosx/x86_64/Swift.swiftmodule

Argh.

Building the compiler is not a quick thing. Even a ninja build is a non-trivial investment of time. So if you want to be completely honest, the total coding and build time a lot longer than just five hours. A lot longer.

Make sure you’ve updated your repo, incorporated the latest changes for swift and all of its support tools, and built them all before working on your branches. It will save you a lot of frustration.

Be aware that in the time it takes to create even a small project, you’ll probably be out of date with master. Hopefully this won’t affect your changes, and the files you think you’re patching are still the right files you should be patching.

Designing My Changes

The best way to add anything to the Swift compiler is to find some construct that has already been contributed and look through pull releases to discover what parts of the language they had touched.

That’s how I got started with my assertions/optimization revision. I looked at the recent canImport()pull, and targeted the seven files that involved. In the end, I only  needed to modify four files in total, excluding tests. It was a fairly “clean” and simple update.

To add #dogcow, again excluding tests, I had to change nearly two dozen files, most of them written in C++, a few using Python and Swift’s own gyb (aka “generate your boilerplate”) macros.

I’ve put up a gist that details my notes as I performed file edits. (I did have to make some further changes once I started testing.) Each group consists of a file name followed by my changes, with some context around them to make it easier to return to those parts of the file.

That’s followed by any relevant grep results that encouraged me to edit the file in question plus error messages from the compiler, of which there were a few, as I made several typos and forgot at times to add semicolons to the ends of lines. (Damn you semicolons! shakes fist) I put ### markers near the errors to make them easier to find in the notes.

As you walk through my notes, you’ll notice that I had to create a token (pound_dogcow), which is a kind of MagicIdentifierLiteralExpr expression. By inserting a simple token without arguments and returning a string, I cut down on my need to parse additional components or produce a complicated return value.

(Sorry Dave! I’ll get there I hope… After all, I know where each of the five components of Context live: file, line, column, function, and dsohandle. I just don’t know how to build and export the struct so that it gets put into place and can be consumed by the Swift user.)

As a string, my #dogcow can be used as a literal, so I conformed it to KnownProtocolKind::ExpressibleByStringLiteral. It needed to be serialized and deserialized, emit its custom string, support code completion, and more. Scrolling down my file, you’ll see the scope of notes including searches, comments, and edits for this one change.

Debugging

One of the most interesting things that happened during this exercise was when I made an actual logic error, not a syntax error, so Swift compiled but my program failed:

Assertion failed: (isString() && "Magic identifier literal has non-string encoding"), function setStringEncoding, file /Users/ericasadun/github/Apple/appleswift/swift/include/swift/AST/Expr.h, line 1052.

For the longest time I was convinced (wrongly) that because I was using Unicode, that I had somehow screwed up the string encoding. This was actually a coding mistake, an actual bug, and had nothing to do with the way my string was built nor the fact that I used emojis. It took a while to track down because my head was in the wrong place.

Notice I have DogCow returning true here. I accidentally swapped the two lines so it was originally returning false, falling into the Line/Column/DSOHandle case.

 bool isString() const {
   switch (getKind()) {
    case File:
    case Function:
    case DogCow: // it's a string!
      return true;
    // it should not have been down here
    case Line:
    case Column:
    case DSOHandle:
      return false;
  }
  llvm_unreachable("bad Kind");

Proof-of-Concept

Once compiled, I used a few simple calls to test my work. Here’s the source code I used. I accidentally added an extra space in the assignment test. You can see in the screenshot as well:

// String interpolation and default argument
func hello(_ greetedParty: String = #dogcow) {
    print("Hello \(greetedParty)")
}

hello()
hello("there")

// Use in assignment
let aDogCow = #dogcow
print("The value is ", aDogCow)

// Use directly in statement
print(#dogcow)

Lessons Learned

Having built a working version of the compiler incorporating my solo changes, no matter how trivial and yes it was extremely trivial, has been a big confidence builder. Exploring the process from consuming tokens to emitting intermediate language representations has enlightening.

  • I learned to update everything and build from scratch before starting my work. Because if you don’t, you’ll end up doing it later and wasting that time.
  • I learned how to track down similar modifications and use them as a template for exploring what parts of the compiler each change touched.
  • I learned that some errors would not be in the compilation but in the testing, as one tends to forget things like “just because it built doesn’t mean it will compile correctly” when one is very very focused on getting things to run and extremely new to the process.

I have now worked on two (technically three) compiler modification projects. Each has  taught me something new. If you’d like, take a peek at some explorations I’ve pushed to my forked repo:

The DogCow changes are clean, in the style of something that I might actually do a pull request for. The optimization checks are not. They retain all my little in-line notes I use for searching through text files to find what I’ve changed.

The early debug checks represent the time before I could get all the compiler tools built on my system. I was basically programming in my head at that point, guessing what would work or not, before the conversation on Swift forums moved me to my current design.

My guesswork was wrong. I focused on using a trio of built-in functions (like _isDebugAssertConfiguration) mentioned on-list. This turned out to be a non-viable solution. I needed to follow the example set by canImport to set my flags.

Finally, a word to the wise: Don’t ./utils/build-script -R in one terminal window and ninja swift in another at the same time. Oops.

Today, playing “No. Yes.”

No.

// So I have a [String: [String]]
// Is there a way to do this without the force unwrapping?
if let _ = result[key] {
    result[key]!.append(decodedValue!)
} else {
    result[key] = [decodedValue!]
}

Yes.

if let decodedValue = decodedValue {
    result[key, default: []].append(decodedValue)
}

Commentary

Dictionaries now support default values (SE-0165 and SE-0154), but even if they didn’t, you could still use nil-coalescing without introducing the check on result’s lookup:

// a [String] type for the rhs is inferred by context
result[key] ?? []

The decodedValue should be handled through with either guard-let for early exit or if-let to unwrap then use. Since this kind of code often appears in loops, if may be the better choice, allowing each optional value to be tested and added.

If you are looping, consider applying a flatMap compactMap to your source array or using for case let, enabling you to unwrap and select your input:

for value in decodedValues.compactMap({ $0 }) { 
    result[key, default: []].append(value) 
}

for case let value? in decodedValues { 
    result[key, default: []].append(value) 
}

The two approaches are not significantly different, although I tend to lean towards the latter. It involves a single pass and I think it’s more elegant.

Cleaning up doc comments for formatted commits

I’m working on a proposal to introduce CountedSet, cousin to NSCountedSet, to Swift. This kind of type involves a massive amount of doc comment content. I decided to adapt the comments from Cocoa Foundation (for NSCountedSet) and Swift Foundation (for Set) as part of my coding and quickly found how ridiculous it was to do this by hand.

At first I tried to write an Xcode “reflow doc comments” extension but as I found in the past, Xcode extensions are a dreadful pain to program and debug. It really wasn’t worth doing this (although it would be my preferred workflow for use) in terms of spending my time well.

Instead, I decided to create a simple playground. I’d paste my Swift file into a known Resources file (in this case, test.swift, although I’m sure you can come up with a better name than that if you use this). I’d process the text with a simple playground implementation and print to stdout.

It was an interesting problem to solve and one that took slightly longer than I anticipated. It’s also one that’s only partially complete. The log jams involved looking ahead at the next line to decide when each blob of text was complete so it could be reflowed, preserving paragraph breaks in the comments, respecting code blocks, and leaving any in-file code intact. Reflowing the words was much easier. I’m sure you’ve written that part of it in any number of algorithms and intro-language classes.

The parts I didn’t tackle were the special formatting required for doc comment keywords, like - Parameter, - Returns, - SeeAlso, and so forth. The associated lines for these items must be reflowed with proper indentation so the Quick Help parser can properly parse them. I leave that for another day because they are relatively minor work compared to reflowing long and complex doc comments as a whole.

I’ve put my code up on Github if you want to offer improvements, fixes, or feedback:

 

Making number checks pretty with unbounded ranges and formatting

This code showed up in a discussion yesterday:

switch averagePosts {
        case 10000000...25000000: return .d
        case 5000000...10000000: return .c
        case 2500000...5000000: return .b
        case 500000...2500000: return .a
        case 250000...500000: return .b
        case 100000...250000: return .c
        case 50000...100000: return .d
        default: return .f
}

I thought it would be much cleaner and easier to use if the coder incorporated unbounded ranges and used Swift’s built-in underscore formatting for their numbers:

switch averagePosts {
case 10_000_000...: return .d
case  5_000_000...: return .c
case  2_500_000...: return .b
case    500_000...: return .a
case    250_000...: return .b
case    100_000...: return .c
case     50_000...: return .d
default:            return .f
}

A few things to note about this rewrite:

Inserting the underscores and aligning the numbers greatly simplifies code validation. Normally I’m not a proponent of making code look “pretty” by adding spaces. Here it serves a very specific purpose, allowing eyeball checks of each number with respect to its fellows.

I kept colon-hugging with default but spaced out the return value to provide a uniform return column for the same reason.

My logic is slightly different from the original as I don’t check for numbers exceeding 25 million. If you wanted to add that, the case would have to be 25_000_001 and not 25_000_000 to exclude the valid value that returns .d. returning .f (the default) instead.

This is a good example of where a full set of Swift range operators would help. Using 25_000_000<..., which would be basically the unbounded half-open-at-the-start operator, could exclude 25 million without using the uglier “I have to add one” hack, which is an example of “have to stop and think” code, which is best avoided.

A full set of operators like those I mentioned in this post would have to be updated for Swift’s new support of unbounded ranges:

  • <... and ...< half open unbounded
  • ... closed (2 values) or unbounded (1 value) or all numbers of type (0 values)
  • <.. and ..< half open, bounded
  • <.< fully open, bounded

Swift Evolution and Civility

Congratulations to Chris Eidhof for all his work shepherding SE-0199 through Swift Evolution. The proposal adds a mutating toggle function for Boolean values to reduce in-code redundancy and enhance readability.

I’m concerned by some un-collegial reactions to its acceptance, which seem to boil down to “this is too trivial a change to the language” stated through sarcasm and ad-hominem attacks: on Twitter, on the evolution forums, and even a mean-spirited Github repository.

If you are passionate about Swift, I urge you to actively participate in the public review and comment process on forums.swift.org. This proposal passed with overwhelming support and the core team has encouraged similar proposals to fill gaps in the standard library.

The curious case of operator assignment

Fire up a playground or skeleton project, add this code, and attempt to compile it.

let lessThan = <

It won’t compile, and your error should read something along the lines of “unary operator cannot be separated from its operand”. Jared Sinclair discovered this issue the other day when trying to work indirectly with comparison operators.

It’s an odd error message given that the default declaration of  < is infix, not prefix. And he and I spent a bit of time the other day tracking down why this error happened and how to work around it.

To give a larger context, take a look at the following snippet:

struct Foo: Comparable {
    static func fooify(_ string: String) -> String {
        return "foo"
    }
    static func ==(lhs: Foo, rhs: Foo) -> Bool { return true }
    static func <(lhs: Foo, rhs: Foo) -> Bool { return true }
}
typealias Fooquatable = (Foo, Foo) -> Bool

This struct defines three static functions: fooify(_), ==, and <. However, you can access only the fooify member by name:

let foofify = Foo.fooify
let lessThan = Foo.< // use of unresolved operator '.<'

So what do you do if you want to pull some Fooquatable static member into an expression? The answer is to both parenthesize and type the operator:

// Parenthesized and typed
let lessThan: Fooquatable = (<) // works 
let fooperations: [Fooquatable] = [(==), (<)] // Works

// Untyped or non-parenthesized
let fooperations = [(==), (<)] // ambiguous use of operator '=='
let fooperations: [Fooquatable] = [==, <] // expected expression after unary operator
let lessThan = (<) // Ambiguous use of operator '<'

This behavior raises several interesting questions: why does Xcode not see or offer to autocomplete static operator implementations, why does it produce the “unary” error message for an infix operator, and why are those parentheses needed?

Xcode does not, at least at this time, include static operator implementations in its completion list. Member operators cannot be referenced using dot syntax (SR-7155). Autocomplete excludes both < and == and you cannot refer to Foo.< or Foo.== in code.

Swift’s “unary” error messages seem to derive from its tokenizer (SR-7131):

if (OperEndLoc == Tok.getLoc())
 diagnose(PreviousLoc, diag::expected_expr_after_unary_operator);
 else
 diagnose(PreviousLoc, diag::expected_prefix_operator)
 .fixItRemoveChars(OperEndLoc, Tok.getLoc());

It’s built in that after the previous operator’s end location, there is a space so it goes to the else clause. A standalone < or == following an assignment wants another token for unary application, which it does not get, rather than treating this as a static method reference. The remove-characters fixit somewhat pointlessly deletes any comment and newline that follows the operator.

The parentheses bypass ambiguity. Adding parentheses, whether in a standalone assignment or as part of a collection of operators, allows the compiler to recognize the operator as a static method. Explicit typing is still needed since you can’t refer to Foo.<.

Jared Sinclair and I tried a lot of workarounds beyond parentheses that all failed:

  • Foo.< did not work, even though operators are not Swift sugar and are full static members.
  • Foo.(<) failed as invalid syntax.
  • Foo[keyPath: \<] and its parenthesized variations were also failures. (“expected expression path in Swift key path”)
  • < as a standalone operator failed as the compiler was unable to resolve adjacent operators. Attempting to manipulate relative precedence had no effect. The compiler never interpreted < as an expression.
  • You cannot try to game the system with “clever” workarounds like: let lessThan: Fooquatable = { return <}() or let lessThan: Fooquatable = { return <; }().
  • (<) worked because the parentheses created an explicit expression value, so there was sufficient disambiguation for the compilation to proceed.

Have any further thoughts? Drop a note and let me know or add to the comments at bugs.swift.org.

Update: Mark Lacey has been working on something for “operators as members“.