Erica Sadun

Where technology meets something or other

Archive for July, 2016

Chris Lattner on wrapping up Swift 3, starting Swift 4

Chris Lattner writes on Looking back on Swift 3 and ahead to Swift 4.  Key tl;dr points:

  • Open source is great but you can’t make everyone happy.
  • Swift 3 in Fall 2016. Swift 3.x will release in Spring 2017. Swift 4 in Fall 2017, not counting minor releases for bugs/compatibility (e.g. 3.0.1)
  • Swift 4 must deliver its source stability promise, plus resilience, ABI, improving generics, strings, etc.
  • Syntactic sugar can wait and are the lowest-priority items.
  • Scheduling is hard. Goals aren’t promises of delivery. Scheduling and planning from the start will take priority.

Here is the message in full:

Hello Everyone,

The Swift 3 release is nearing completion, so it is time to look back on the release, learn from what happened, and use it to shape what we (the Swift community) do in the year ahead. Overall, Swift 3 is going to be an absolutely amazing release, and it is impressive how much got done. Thank you to everyone who contributed to making it happen. Instead of diving into a flurry of new proposals immediately, it is important to take stock of where we are, and look at the bigger picture.

Metapoint: this email is ridiculously long and covers multiple topics. Instead of replying to it directly, it is best to start new threads on individual topics that you’d like to discuss. Just tag them with “[Swift 4]” in the subject line.

Swift 3 Retrospective

Every year of Swift’s development has been completely different from the previous one, and I expect Swift 4 to continue this trend. With a goal of learning and improving year over year, here are some observations & retrospective about Swift 3:

  • Open source is pretty great. It has been incredible to see such a vibrant community working so well together, and to see you come together practically overnight. It is really fantastic to work with such a talented and enthusiastic group of people!
  • Open source also brings challenges. I think it is fair to say that “open design” is slower and less predictable than “closed design”. However, the end result is significantly better, and therefore the tradeoff is worth it. A big “Thank you!” goes out to everyone who helps improve Swift through the evolution process.
  • Software scheduling (particularly with open source) continues to be difficult-to-impossible to predict. We set off on Swift 3 with a lofty set of goals that had to be trimmed back later. It is good to have high goals, but we need to do a better job of communicating that “goals” are not “promises” so that people don’t feel misled.
  • The community benefits from keeping focus on a limited number of topics, because if there is too much going on, no one can follow and keep track of it all. It is important for the core team to be involved in key discussions up front. In the Swift 3 cycle, it was problematic that many folks had no time to follow the threads until after the review period completed.
  • Having clear goals is liberating. In the December/January timeframe in particular, we loosely scoped what sorts of ideas would fit into Swift 3, and embarked on several projects that ended up being far beyond what we could handle. Later in the release, we had very specific goals (for example, “no additive proposals”) which made it much easier for everyone to focus on what was important.
  • It is impossible to make everyone happy, particularly when discussing which features to pick up and prioritize* because that implicitly de-prioritizes something else. This is inevitable though, because there is no way that all the interesting work will fit into a one-year release cycle. Fortunately, there will “always be another release,” and each new release will have its share of big improvements.

With that as context, let’s look ahead!

Swift Release Plan

Over the next year, the core team expects to ship two major releases of Swift: Swift 3.x in Spring 2017 and Swift 4 in Fall 2017. In addition to the major releases, we will surely ship some minor releases (e.g. Swift 3.0.1) to fix bugs or service the needs of the corelibs or other swift.org projects.

Swift 4 Release Cycle Planning Approach

From our experience with Swift 3, we know we need to pick and choose what we’re going to tackle. For Swift 4, the primary goals are to deliver on the promise of source stability from 3.0 on, and to provide ABI stability for the standard library. As such, the core team has decided to take a two-stage approach to the next year:

Stage #1: Focus on the essentials required for Source and ABI stability, and keep reasonably strict focus on only that work. This means that any features that don’t fundamentally change the ABI of existing language features or imply an ABI-breaking change to the standard library will not be considered in this stage. For example, generics features like conditional conformance is an additive feature, but since it is expected to reshape much of the standard library it would be part of Stage 1. On the other hand, language support for regular expressions would not affect existing ABI nor cause major changes to existing standard library features, so it would not fit in Stage 1.

The work entailed by Stage 1 is non-trivial (explored in somewhat more detail below), and will probably keep us busy until the Spring.

Stage #2: As the design and implementation work on the Stage 1 features is cresting, we’ll scope and plan a few other large features based on how much time we have left. I’m optimistic that we’ll have time to pick up some of the long list of features explored below, but we won’t know which ones those are until we know how much time we have to develop them with.

In addition to new features, we also need to reevaluate accepted source breaking proposals that didn’t make it into Swift 3. These proposals will not necessarily be grandfathered in* we will need to evaluate them against the goals of Swift 4 and decide what to do with each of them on a case by case basis.

Finally, while it isn’t specifically related to swift-evolution, I want to call out quality & performance work. The core team wants to continue to improve quality, including fixing compiler bugs and improving error and warning diagnostics. Performance is also an ongoing area of critical development, including improving the performance of the generated code, improving the standard library implementation, speeding up compile time, etc. All of this work can happen in either stage.

Swift 4 Stage 1 Goals

With a focus on source and ABI stability, the core team had an initial discussion about what makes up the Stage 1 effort. Here are the features that we are prioritizing for Stage 1:

  • Source stability features: These should be relatively small, but important. For example, we need a “-std=swift3” sort of compiler flag. We may also add a way to conditionally enable larger efforts that are under development but not yet stable* in order to make it easier to experiment with them.
  • Resilience: This provides a way for public APIs to evolve over time, even in the face of ABI stability. For example, we don’t want the C++ “fragile base class” problem to ever exist in Swift. Much of the design and implementation work was done in the Swift 3 timeframe, but there are still major missing pieces, including the user-visible part of the model (e.g. new attributes).
  • ABI Details: There are a ton of small details that should be audited and improved in the code generation model. This is mostly relevant to swift-dev, and not specifically a swift-evolution topic.
  • Generics improvements needed by the standard library: I expect conditional conformances to be the very top of this list, and for recursive protocol requirements and more powerful associated type constraints to be close followers. However, the standard library gurus need to break down what is absolutely essential to finally eliminate the rest of the “_” protocols and manifest the public API of the standard library in the right way for the long term.
  • String re-evaluation: String is one of the most important fundamental types in the language. The standard library leads have numerous ideas of how to improve the programming model for it, without jeopardizing the goals of providing a unicode-correct-by-default model. Our goal is to be better at string processing than Perl!
  • Memory ownership model: Adding an (opt-in) Cyclone/Rust inspired memory ownership model to Swift is highly desired by systems programmers and folks who want predictable and deterministic performance (for example, in real time audio processing code). More pertinent to the goals of Swift 4, this feature is important because it fundamentally shapes the ABI. It informs code generation for “inout”, how low-level “addressors” work in the ABI, impacts the Swift runtime, and will have a significant impact on the type system and name mangling.

Each of these areas has had some thought put into them already, but are still a long way from being formal proposals. I expect and hope that they will turn into major discussions early in the Swift 4 cycle. Further, since we haven’t fully scoped what impacts ABI stability, there may be other specific additions as we learn more. Finally, it is also possible that we may choose to scope in specific small features that are high value to unblock SwiftPM or other swift.org projects.

Possible Swift 4 Stage 2 Efforts

As I mentioned above, at this point it is impossible to know what can be achieved in the Stage 2 timeframe, because we don’t know how long that timeframe is. The core team would also like to converge Swift 4 development earlier in its cycle than Swift 3 did, in order to fix more bugs late in the release and provide a longer bake time.

That said, I’m optimistic that we’ll be able to pick up and tackle some of the commonly requested new features. To give you an idea of some of them, here is a list. Please note that this is not a plan or commitment, it is just a laundry list of commonly requested features:

  • Reflection: The core team is committed to adding powerful dynamic features to Swift. For example, Swift 3 already added nearly all the infrastructure for data reflection (which is already used by the Xcode memory debugger). We should use this infrastructure to build out a powerful user-facing API. Similarly, we would like to design and build out the implementation for dynamic method reflection runtime + API support.
  • First class concurrency: Actors, async/await, atomicity, memory model, and related topics. This area is highly desired by everyone, as it will open the door for all sorts of new things on the client, server and more. We plan to start formal discussions about this in Phase 2, but it is unfortunately crystal clear that a new concurrency model won’t be done in time for the Swift 4 release. This is simply because it will take more than a 12 months to design and build, and we want to make sure to take time to do it right. It also makes sense for the memory ownership model to be better understood before taking this on.
  • Generics improvements: The generics manifesto includes many exciting enhancements to the generics system, many of which will not be specifically required for ABI stability of the standard library, but would make Swift generics more powerful and expressive.
  • .swiftmodule stability: At some point we need to stabilize the “.swiftmodule” binary file format (or replace it with a different mechanism) to allow 3rd party binary frameworks. This is a very large amount of work over and above what is required for ABI stability of the standard library.
  • New scripting features: Regular expressions, multi-line string literals, etc. Having these would make Swift much more appealing to the crowd doing scripting and those building web technologies, among others. They’d also help round out the String model.
  • Property behaviors: This feature promises to provide powerful abstractions over the existing property model. The deferred SE-0030 proposal describes this opportunity well.
  • So many others: Submodules, implicit promotions between numeric types, importing C++ APIs, hygenic macro system, guaranteed tail calls, making enums enumerable, typed ‘throws’, user defined attributes, abstract methods/classes, better SIMD support, ‘dynamic’ for non-@objc, data parallelism support, higher kinded types, …
  • Syntactic sugar: I won’t list them all, but there are a ton of other trivial to small proposals that frequently come up, typically things seen in other languages that solve specific problems. These are the lowest priority to tackle for Swift 4.

So there you go, an overly long email laying out some thoughts and ideas of what to do over the next year. One specific thing to keep in mind is that Swift 3 isn’t done yet. While source breaking changes are (nearly) done, there is still time for bug fixing and other quality work, and that is very important to the release.

I think it would be helpful for us to spend the immediate future discussing the general parameters of how we tackle the releases over the next year, then iron out the Stage 1 features conceptually. We should only start writing proposals when the specific design of them are well understood. The core team does not want to be in a situation where there are so many proposals floating around that we can’t keep track of them, or that they prevent tackling the large and important high priority projects in front of us.

Thank you. Again, please start new threads if you’d like to dive in more deeply into specific topics

The curious case of the “optional requirement” in the night

Has this happened to you? You write code and the compile warns you that a method “nearly matches” an “optional requirement”.

Screen Shot 2016-07-28 at 1.37.18 PM

Let me tell you why this happens and what you should be looking for.

First, some sample code:

public class Foo: UIViewController {
}

extension Foo {
    
    public func tableView(_ tableView: UITableView, 
        numberOfRowsInSection section: Int) -> Int {
        return 1
    }
    
    public func tableView(_ tableView: UITableView, 
        cellForRowAt indexPath: IndexPath) -> UITableViewCell {
        return UITableViewCell()
    }
}

This compiles. No warning. If you add the missing UITableViewDataSource conformance declaration, it still compiles without warnings. But if you add conformance to UITableViewDelegate to the extension without conforming to data source, you get this odd error message.

That’s because the “near miss” detector code sees the delegate conformance, and tries to match the method declarations to the known conformance. Both methods here are public, there’s an overlap in the naming pattern, and the detector assumes you’re most likely trying to conform to the protocol named in the extension.

If in fact,  you’ve forgotten a conformance, just add it in. Declaring both data source and delegate fixes this issue. If you’re encountering a weird accidental overlap of names, you can:

  • rename the method so it doesn’t trigger the detector
  • move the method to a different extension
  • mark the method private, so it’s clear that you’re not using it for conformance

Of course, if  you really did mess up the name and it’s not an accidental overlap, you can just fix your error.

Dear Erica: How do I mimic Objective-C’s delegate inheritance pattern?

Kyle: “Is it possible to have class Foo: Bar where Foo has a delegate of type protocol FooDelegate: BarDelegate where Bar also has a delegate declared as type BarDelegate?  In my case i am subclassing scrollview and want to declare delegate as my own type that conforms to UIScrollViewDelegate? I get the error that a property delegate with type FooDelegate? cannot override a property with type UIScrollviewDelegate?”

If I’m understanding, you want to be able to create delegation where the same property (delegate) can be assigned to ever more specialized protocols when subclassing. So, for example, you might have a base class like UIScrollView whose delegate is UIScrollViewDelegate, and a subclass like UITableView whose delegate is UITableViewDelegate. Right?

In Swift, you must ensure the child protocol conforms to the parent protocol. I do not believe is the case with the scroll and table view delegates in Objective C. Start with a core delegate protocol like this.

public protocol DelegateProtocol: class {
    func showMyType() -> Void
}

It’s an empty placeholder for all delegation protocols. You don’t need the showMyType requirement here. I’m just putting it in for demonstration, so you can check where the required member is implemented. If you want to use weak delegation, you must declare class, as in this example.

To demonstrate how this works, here are a core delegate protocol and a derived one:

public protocol CoreTypeDelegate: DelegateProtocol {}
extension CoreTypeDelegate {
    public func showMyType() { print ("This is a Core Type Delegate (required)") }
    public func shared() { print ("Shared at Core level (extension)") }
}

public protocol DerivedTypeDelegate: CoreTypeDelegate {}
extension DerivedTypeDelegate {
    public func showMyType() { print ("This is a Derived Type Delegate (required)") }
    public func shared() { print ("Override at Derived level (extension)") }
    public func exclusive() { print ("Implemented only at Derived level (extension)") }
}

// Implement one of each
class ACoreDelegate: CoreTypeDelegate {} // like UIScrollViewDelegate
class ADerivedDelegate: ACoreDelegate, DerivedTypeDelegate {} // like UITableViewDelegate

print("-- Core delegate")
let myCoreDelegate = ACoreDelegate()
myCoreDelegate.shared() // core version
myCoreDelegate.showMyType() // core version

print("-- Derived delegate")
let myDerivedDelegate = ADerivedDelegate()
myDerivedDelegate.shared() // derived override
myDerivedDelegate.showMyType() // derived version
myDerivedDelegate.exclusive() // only in derived

These three methods differentiate how required members, extension-only members, and exclusive-members are accessed when instances are used in different roles.

Next, here’s a basic protocol for classes that use delegates:

public protocol Delegatable {
    associatedtype DelegateType: DelegateProtocol
    var delegate: DelegateType? { get set }
}

This protocol consists of a conforming type declaration, and a delegate property. This indirection allows you to store instances of arbitrary types, so you can use the same delegate property for both a base class and its more specialized children.

To see this in action, you need delegatable types . Here’s a base class, similar to the role that scroll views play. (Warning: weak delegation will crash playgrounds. If you’d rather use a playground, omit weak in the property implementation.)

// Like UIScrollView
public class BaseClass<Delegate: CoreTypeDelegate>: Delegatable {
    public typealias DelegateType = Delegate
    public weak var delegate: DelegateType? = nil
    public func baseFunc() { print("implemented in base class") }
}

Now you can create an instance, specifying a base class for delegation:

var myBaseInstance = BaseClass<ACoreDelegate>()
myBaseInstance.delegate = myCoreDelegate // ok
print("-- Should use core implementations")
myBaseInstance.delegate?.showMyType()
myBaseInstance.delegate?.shared()

The derived delegate class inherits from the base delegate class. You can substitute it in, but the instance will still use core implementations.

print("-- Conformance by derived delegate type")
print("is CoreTypeDelegate", myDerivedDelegate is CoreTypeDelegate) // true
print("is DerivedTypeDelegate", myDerivedDelegate is DerivedTypeDelegate) // true
print("is ACoreDelegate", myDerivedDelegate is ACoreDelegate) // true
print("is ADerivedDelegate", myDerivedDelegate is ADerivedDelegate) // true

print("-- Will still use core implementations, because it's typed to CoreTypeDelegate")
myBaseInstance.delegate = myDerivedDelegate // ok
myBaseInstance.delegate?.showMyType()
myBaseInstance.delegate?.shared()

print("-- Casting delegate, only required implementation does not override")
guard
    let derived = myBaseInstance.delegate as? DerivedTypeDelegate  else {
        fatalError("Cannot cast derived delegate to DerivedTypeDelegate")
}
derived.showMyType() // still uses core showme
derived.shared() // uses derived shared
derived.exclusive() // not available in core

Subclassing the base class and creating instances with the derived delegate protocol enables you to use the delegate property with a more specialized feature set.

// Like UITableView
public class DerivedClass<Delegate: DerivedTypeDelegate>: BaseClass {
    public func derivedFunc() { print("implemented in derived class") }
}
var myDerivedInstance = DerivedClass<ADerivedDelegate>()
// myDerivedInstance.delegate = myCoreDelegate // no, does not allow, cannot assign value of type 'ACoreDelegate' to type 'ADerivedDelegate?'
myDerivedInstance.delegate = myDerivedDelegate // yes

print("-- Should use native implementations for all")
myDerivedInstance.delegate?.showMyType() // yes!
myDerivedInstance.delegate?.shared()
myDerivedInstance.delegate?.exclusive()

As always, I’ve probably messed up some things along the way, so if you have better solutions or you find an issue in this example, please let me know and I’ll fix. Thanks!

End of Source Breaking Changes for Swift 3

Ted Kremenek writes: “Today is July 27 — and the last planned day to take source-breaking changes for Swift 3. It has been an incredible ride to this point, so let’s take stock of where we are.”

Accepted but not implemented reviews:

Ted continues:

“These are all changes the community has approved for Swift but did not make today’s cutoff. Some of these proposals have implementations actively underway. For those proposals already in active development — and near completion — I am okay with extending the deadline for those changes to Friday, July 29. Such changes need to be approved by the release manager (myself) and should be merged into master via a pull request. When creating the pull request, please assign it to me (tkremenek), and mention the pull request on the swift-dev mailing list as well with the SE number in the email title.

The rest of the unimplemented proposals do not make Swift 3. This leaves us with the question of what to do with them. These proposals represent the known and reviewed changes we want to make to Swift, but inevitably there will also be changes that we don’t even know about today that we will want to take into Swift that can impact core source stability. That said, we also have a very strong desire to maintain source compatibility with Swift 3 and Swift 4 as much as possible to provide some stability for which Swift users to build upon. The challenge of course is reconciling these diametrically opposing goals: maintaining source stability while having the ability to incorporate more core (and important) language changes that are possibly source-breaking.

The Swift team at Apple has reflected on this and decided what it “means” for Swift 3 to be source compatible with Swift 4 and later releases going forward. Our goal is to allow app developers to combine a mix of Swift modules (e.g., SwiftPM packages), where each module is known to compile with a specific version of the language (module A works with Swift 3, module B works with Swift 3.1, etc.), then combine those modules into a single binary. The key feature is that a module can be migrated from Swift 3 to 3.1 to 4 (and beyond) independently of its dependencies.

While the exact details of how we will accomplish this feat are still being discussed, here is a sketch of how this will likely work in the Swift 4 timeframe. The key enabler is a new compiler flag that indicates the language version to compile for (e.g., similar to the clang -std=c99 flag). The compiler flag will be provided by the build system you are using (e.g., Xcode, SwiftPM, etc.) on a per-module basis:

For language syntax/semantics, the compiler can use the language mode to properly implement the language version being used by a module.

For the Standard Library, additive and subtractive changes are easily handled (the former by just adding them, the later by using deprecation techniques). For semantics changes, things are much more complicated, and will need further study.

The great thing about this approach is that a single Swift 4 compiler is building all of the sources in an application. This allows us to roll out this approach before achieving full ABI stability — something that will be a goal for Swift 4, but is impractical to achieve for a Swift 3.x release. It also provides us a general framework in the future for handling source compatibility as Swift evolves.

To make this more concrete, suppose an application is written to use Swift 4, but uses packages via SwiftPM that are written using Swift 3. A single compiler would build both the app and the packages — thus ensuring that all the compiled sources are binary compatible. It would not be the case that a framework built with the Swift 3 compiler could be used by an app built using the Swift 4 compiler. That kind of library binary stability (ABI) will be a key goal of the Swift 4 release.

These constraints mentioned above will serve as scaffolding for Swift 4 development. Discussion about Swift 4 commences on Monday. Ahead of that, Chris Lattner plans to send out thoughts from the Core team on some of the known key goals (and non-goals) for the release. In the meantime, the focus over the next couple days should be taking stock of what has landed for Swift 3 and to see if any of the proposals mentioned above are close to being completed or are truly out of scope.

Thank you again to everyone for making Swift 3 such as fantastic release!”

Very last second Swift 3 reviews: big changes, little time

It’s crunch time. Several reviews are running concurrently for just a couple of days. All breaking changes must wrap up in the next week and Swift 4 design starts on 8/1.

Last Second Reviews

SE-0129: Package Manager Test Naming Conventions Accepted. Team writes:

There was relatively little feedback on the proposal but it had unanimous support, along with additional support for the future directions that are unblocked by the proposal with regard to allowing additional target types under the `Tests` convention directory.

SE-0131: Add AnyHashable to the standard library in review to 7/25. Accepted  “Currently the Objective-C type NSDictionary * is imported as [NSObject : AnyObject]. We used NSObject as the key type because it is the closest type (in spirit) to AnyObject that also conforms to Hashable. The aim of SE-0116 is to eliminate AnyObject from imported APIs, replacing it with Any. To import unannotated NSDictionaries we need an Any-like type that conforms to Hashable. Thus, unannotated NSDictionaries will be imported as [AnyHashable : Any]"

Team writes:

The feedback on this proposal was quite positive.  A few questions were raised, but were answered on-thread.  Thank you to Dmitri Gribenko for writing this proposal and driving this discussion forward.

SE-0132: Rationalizing Sequence end-operation names in review to 7/26 is Deferred from Swift 3. “Sequence and Collection offer many special operations which access or manipulate its first or last elements, but they are plagued by inconsistent naming which can make it difficult to find inverses or remember what the standard library offers. We propose that we standardize these names so they follow consistent, predictable patterns.” Team writes:

This is a very large proposal very late in the Swift 3 schedule. It has some clearly good pieces to it, but also many parts that are controversial with the community, and requires more design and iteration than time permits. It is best to defer this and make a change when we have something truly great, giving the benefit of proper time to design and evaluate the proposal.

Thank you to Brent Royal-Gordon and Dave Abrahams for driving this discussion. When the goals and parameters of Swift 4 are established, we should pick up this thread and see what makes sense to do here.

SE-0133: Rename flatten() to joined() in review to 7/26 Accepted. “This proposal renames flatten() to joined() (with no separator argument). It also adds a default separator of "" to the String-specific version of joined(separator:).” Team writes:

This proposal had significant positive community feedback for aligning common operation names, but raised questions about whether ‘flatten’ was term of art, and what it would mean for related operations like ‘flatMap’. The core team discussed both sides of this debate, and decided it is best to rename ‘flatten’ to ‘joined’, but keep ‘flatMap’ as it is, to preserve its term of art. The core team prefers that it remain a distinct overload of joined(separator:) to preserve performance. It also requests that the returned collection types (FlattenCollection and friends) be left as-is for now, since they are not names commonly directly referenced, and we’d like to keep the change minimal.

SE-0134: Rename two UTF8-related properties on String in review to 7/26 Accepted with Revision. “This proposal renames nulTerminatedUTF8 and nulTerminatedUTF8CString to enhance clarity and reduce mismatch between user expectations and the Swift programming language.” Team writes:

This proposal has two parts. The core team approves the first part: “Rename nulTerminatedUTF8CString to utf8CString”. This property is intended for use when interoperating with APIs that take C-style “char *” parameters.

For the second part, the proposal suggests “Rename nulTerminatedUTF8 to nullTerminatedUTF8.” The core team discussed this, looked at various uses of the API (among code bases publicly available on github) and came to the conclusion that it is best to just remove this property outright. The clients we found would be better served using the .utf8CString property.

Last Second Acceptances and Rejections

SE-0122: Use colons for subscript declarations is rejected. Team writes:

The feedback on this proposal from the community was overall divided with a slight bias towards reject.  Reasonable arguments were made on both sides: subscripts are definitely property-like in some ways, but they are also function-like in other ways.  Further, if we were aiming to align declaration syntax with use, arguably the parameter list should be enclosed in square brackets (but to be clear, we’re not going to do that). Overall, the core team agrees with the pervasive sentiment that this is not important enough to make a change for.

SE-0125: Remove NonObjectiveCBase and isUniquelyReferenced is accepted. Team writes:

This proposal had significant community feedback to help refine and improve its design, and Arnold has incorporated that into his v2 of the proposal.  The core team agrees the new revision is a good design.

SE-0127: Cleaning up stdlib Pointer and Buffer Routines is accepted with a modification.  Team writes:

This proposal got a significant amount of helpful feedback, which Charlie has already incorporated into the proposal.  The core team requests one additional minor change, which is to drop the addition of the ObjectIdentifier.unsafeAddress field.

SE-0128: Change failable UnicodeScalar initializers to failable is accepted. Team writes:

Feedback was universally positive, and the proposal is straight forward.

SE-0130: Replace repeating Character and UnicodeScalar forms of String.init is accepted. Team writes:

There was very little public discussion about this proposal, but the core team believes this eliminates a real ambiguity, while leaving doors open for future improved designs.

 

Wrapping up Swift 3

Of these four reviews, SE-0131 and SE-0134 are “gimmes” — that is, minor fixes that need relatively little discussion, offer obvious improvements, and can be evaluated on the spur of the moment. SE-0133 requires a little thought (“A concatenation of the elements of self“), but is also an easy up-or-down vote.

In contrast, SE-0132 is huge. It really deserves more time, attention, and consideration than the Apple-imposed deadline will allow. Brent‘s done a lot of work on perfecting this. If time permitted, it would have benefited from a few more weeks of discussion and revision.

Swift 3 Timeline

Swift 3’s  timeline concerns me. It will be ready for GM release along with Xcode 8 and iOS 10. I, personally, would prefer Swift 3 to be delayed six months or more to get it right the first time. As the planned major “breaking” release, the idea is twofold:

  • Offer incremental adoption by supporting Swift 2.3, which allows developers to build for the App Store using modern APIs in Swift 2.
  • Incorporate breaking code into Swift 3, so Swift 4 is ” as source compatible with Swift 3 as we can reasonably accomplish”. Further: “[S]ource breaking changes will be much more difficult to justify after Swift 3 is out the door, we’re prefer to roll them into Swift 3 where ever possible.”

With just a few days left before it becomes really really hard to fix fundamental design (although, I anticipate that dot-version changes to Swift 3 will be slightly more forgiving than Swift 3-to-Swift 4 changes), there’s been a big rush to find obvious flaws. For example, Open Issues Affecting Standard Library API Stability.

Update: Chris Lattner writes, “I strongly suspect there will be at least one 3.x release.”

On the other hand, August 1 is when the rush of additive proposals that have been back-burnered can come out and play. Swift 3 was all about getting the core language design fixed. Swift 4 is when new features can be considered. I’m hoping for things like method cascades, lenses, a better suite of functional programming tools, macros, strings-on-steroids, namespacing (without enums), and more.

See also: Announcement: Swift 3 Endgame and Swift 4 Kickoff

New Swift Evolution Reviews

Since so many new items have entered review after Wednesday, I’m adding an extra summary post. You can find the weekly full post here.

SE-0130: Replace repeating Character and UnicodeScalar forms of String.init in review to 7/24. “This proposal suggest replacing String initializers taking Character or UnicodeScalar as a repeating value by a more general initializer that takes a String as a repeating value. This is done to avoid the ambiguities in the current String API, which can be only resolved by explicit casting. It is also proposed to remove one of the String.append APIs to match these changes.”

SE-0129: Package Manager Test Naming Conventions in review to 7/26.  “Make the naming of tests be more predictable and more under the package author’s control. This is achieved in part by simplifying the naming conventions, and in part by reducing the number of differences between the conventions for the the Tests and the Sources top-level directories.”

SE-0128: Change failable UnicodeScalar initializers to failable in review to 7/24 “This proposal aims to change some (ones that are failable) UnicodeScalar initializers from non-failable to failable. i.e. in case a UnicodeScalar can not be constructed, nil is returned. Currently, when one passes an invalid value to the failable UnicodeScalar UInt32 initializer the swift stdlib crashes the program by calling _precondition. This is undesirable if one construct a unicode scalar from unknown input.”

SE-0127: Cleaning up stdlib Pointer and Buffer Routines in review to 7/24. “Update the API to match new API guidelines and remove redundant identifiers.”

SE-0126: Refactor Metatypes, repurpose T.self and Mirror in review to 7/24. “This proposal wants to revise metatypes T.Type, repurpose public T.self notation to return a new Type<T> type instance rather than a metatype, merge SE-0101 into Type<T>, rename the global function from SE-0096 to match the changes of this proposal and finally rename current Mirror type to introduce a new (lazy) Mirror type.”

SE-0117: Allow distinguishing between public access and public overridability revision 3 in review to 7/25 ” A public member will only be usable by other modules, but not overridable. An open member will be both usable and overridable. Similarly, a public class will only be usable by other modules, but not subclassable. An open class will be both usable and subclassable. This spirit of this proposal is to allow one to distinguish these cases while keeping them at the same level of support: it does not adversely affect code that is open, nor does it dissuade one from using open in their APIs. In fact, with this proposal, open APIs are syntactically lighter-weight than public ones.”

There’s a good roundup of (unhappy) opinion over at mjtsai.com.

Update: John McCall writes,

I’m resurrecting this thread to point out some minor revisions / clarifications that came up during implementation.

The first is that the accepted proposal contained “temporary” restrictions that required (1) open classes to subclass open classes and (2) open methods to override open methods.  (2) is actually inconsistent with our existing access-control rules about overrides: the overridden method doesn’t even have to be public.  Making a method open in a subclass shouldn’t force you to expose it in your superclass.  Therefore, we have lifted this restriction.  (1) is consistent with the access control rule on subclasses, so it has been conservatively kept; we’re still open to reconsidering this in the future.

The second is that the proposal wasn’t clear about how open interacts with initializers.  There are many things about our current class-initialization design that are unsatisfactory, but we are not going to fix them in Swift 3.  However, as a general matter, the clearest mental model for initializers is that every class provides its own, independent interface for constructing complete-objects / base-subobjects of that class.  The initializers in each class are formally distinct from the initializers in their subclasses, even when they happen to have the same signature; they are not in any real sense “overrides” (even if in some cases we do currently require the “override” keyword).  This is true even for required initializers, which merely state a requirement that all subclasses must provide a complete-object initializer with a particular signature, rather than expressing a real relationship between those initializers at each level.  Furthermore, it is understood that constructing an object of a subclass necessarily involves delegating to that subclass; subclasses must always be able to initialize their portions of constructed objects.  For all of these reasons, it is correct for initializers to not participate in open checking: they cannot be declared open, but correspondingly there are no restrictions on what initializers can be defined in a subclass, even if they have the same signature as an initializer from a superclass.

I have updated the proposal to reflect this.

Devolving your projects

So you moved to 3 and now are suffering from regretsies. Or you’re using Xcode 8 and you want to build a new 2.3 project. Fear not. Here are some tips for all your legacification needs.

Downgrading a Swift 3.0 Project

If you’re feeling especially regretful about moving a project to 3.0, you can downgrade it back to 2.3, but expect work unless you can regress to an appropriate commit. Adjust your build settings to use Legacy Swift (aka Taylor Lautner):

Screen Shot 2016-07-22 at 10.24.46 AM

Building a New 2.3 Project in Xcode 8

Since Xcode 8 doesn’t let you build 2.3 projects, you need to create a 3.0 project. Use build settings to regress to Legacy Swift.

After, make sure to convert all the code back to 2.3. Xcode offers most of the fixits you need for skeleton projects. Be aware that using “Fix all in scope” tends to crash Xcode 8. (27486260).

Screen Shot 2016-07-22 at 10.26.50 AM

Once converted, throw in a for-loop. This ensures that you’re actually running and compiling Swift 2.3 and not Swift 3:

Screen Shot 2016-07-21 at 9.31.42 PM

Importing 2.2 to Xcode 8 as Swift 2.3

If you’re working with an existing 2.2 codebase and are not ready to migrate to a language that has not, as yet, finished its design, you can choose the Swift 2.3 compiler.

First convert the project.

Screen Shot 2016-07-21 at 9.24.41 PM

Next, choose 2.3. The dialog defaults to Use Swift 3, so don’t hit Next until you mentally confirm your choice.

Screen Shot 2016-07-21 at 9.24.47 PM

For many 2.2 projects, you’re ready to rumble.

Screen Shot 2016-07-21 at 9.25.00 PM

Projects that use deprecated APIs and target iOS 10, may need upgrading:

Screen Shot 2016-07-22 at 10.36.19 AM

Explorations into the Xcode Source Editor Extensions underbelly: Part 1

Xcode source extensions are wildly exciting, surprisingly limited, and infuriatingly frustrating to work with. So I thought I’d share some experiences so you don’t have to suffer through some of my issues.

There are bunches of posts around the web on how to create extensions and I really don’t want to dupe their effort. In a nutshell, create a new Cocoa app. Then create a new macOS > Application Extension > Xcode Source Editor Extension target in that app. You’re ready to code.

If you watched the WWDC video, you’ll have seen the whole process of “run the target in Xcode and a new version of Xcode pops up with a darkened label and then you can test your extension”. I’m here to tell you that this approach is a huge steaming pile of crap doesn’t work as well as I’d hoped. What you really want to do is this:

In AppDelegate.swift of your extension/app project, add the following method. It lets you launch the app, install the latest extension and get the hell out of Dodge without using the “Xcode with the darkened label”.

func applicationDidFinishLaunching(_ aNotification: Notification) {
    let alert = NSAlert(); alert.messageText = "Extension installed! Click OK to quit."
    alert.beginSheetModal(for: window) { _ in
        exit(0)
    }
}

In your main extension Swift file (the class defined for XCSourceEditorExtensionPrincipalClass), add this. It lets you watch the OS X console and ensure that your extensions loaded the way you expected them to. Notice that I use NSLog and not print. Always use NSLog, so your output and debug info goes to the system console.

func extensionDidFinishLaunching() {
    NSLog("NAMEOFMYTARGET extensions did finish launching")
}

Set up your Info.plist in a separate editor pane or even a separate window. If you intend to create a multi-part extension (and most people will), you’ll want to build that extension incrementally and slowly add items to your XCSourceEditorCommandDefinitions dictionary. A typical entry looks like this:

Screen Shot 2016-07-21 at 10.20.38 AM

Also, don’t do what I did here. I used “sadun.ExtensionTestbed” because I know this is a throwaway. Assume you may write multiple extension groups and reverse namespace them properly, e.g. “org.sadun.ExtensionName.CommandName”.

Each time you want to test do this:

  1. Build your target.
  2. Open Products, right-click the .app file and show in Finder.
  3. Quit Xcode and run the app.
  4. Launch Xcode.
  5. Open any project that isn’t a playground because Xcode hates me and doesn’t want to properly load extensions for playgrounds. Always start with a non-playground project and *then* open a playground if you want to work with playgrounds.
  6. If the extension is greyed out (and if you follow these instructions they usually aren’t because this is the sequence for El Capitan that appears to be super reliable), check the OS X console. Look for assertion failures, XPC connection failures, and invalidated connections. If you see these, quit Xcode and relaunch. What you want to see there is the “did finish launching” log line.

I know this workflow is a super pain, especially given Apple’s promotion about in-Xcode testing without quitting and relaunching (let alone running the standalone app), but it’s transformed my source editor extension duty cycle from impossible to annoying. A huge win.

Notes:

  • Don’t “clean” your Xcode extension project while running the extension. Ooops.
  • Once the extension crashes, you need to re-run the installer app.
  • I can’t seem to grab a single cursor position, there has to be an actual selection, so my extensions for adding and removing “/// ” for doc comment markup mean you have to select a line, not just move the cursor there. Or maybe there’s a bug that’s screwing this up. I haven’t quite figured this out yet.
  • A lot of my code is super redundant: operate on all lines, operate on all selection lines, operate on selected text. I really need to re-architect a common subclassable system because the actual logic for most plugins is only a few lines or a call-out to another class.
  • Any extension that works on all lines rather than individual selections seems to be more stable in my experience.
  • My experiments trying to create interactive UI elements with extensions hasn’t been going very well, they run in the original application and not in Xcode, the window is behind xcode, they can’t be interacted with while Xcode is running, and you inevitably have to force quit something.

Screen Shot 2016-07-21 at 10.39.39 AM

Here are some things I’ve been messing with. Let me know how your extensions are going for you.

Proterozoic-Oriented Programming: Swift Evolution

It’s Wednesday, so I’ll be filling these in as they pop up:

Accepted

SE-0124: Int.init(ObjectIdentifier) and UInt.init(ObjectIdentifier) should have a bitPattern: label is accepted. Team writes:

There was very sparse commentary, and all of it was positive.  The core team believes this is an “obvious” proposal.

SE-0121: Remove Optional Comparison Operators is Accepted. Team writes:

Feedback has been universally positive from both the community and the core team, because it eliminates a surprising part of the Swift model at very little utility cost.

SE-0120: Revise partition Method Signature is Accepted. (Yay Nate!) Team writes:

There was very sparse commentary, and all of it was positive.  The one requested change from the discussion is to remove @discardableResult, which has already been made to the proposal.

SE-0116: Import Objective-C id as Swift Any type is Accepted. Team writes:

This proposal was very positively received by the community, and garnered a lot of feedback (which has already been incorporated into the proposal).  This proposal has significant implementation complexity, but the work to build it is nearing completion.  When this lands, there is some risk that we’ll need to revise the design in small ways due to unanticipated effects, but the core team believes it is the right overall design to aim for.

SE-0101: Reconfiguring sizeof and related functions into a unified MemoryLayout struct is Accepted. Team writes:

The proposal has been *accepted*, with a revision to change “MemoryLayout” into an enum with no cases.  The overall feedback was very positive.  The core team did discuss a proposal to move the generic type to each member, but felt that that would increase the verbosity of using the API (since T.self would be required) and would lead to possible confusion in the API (because it wouldn’t be clear whether you were asking for the size of a type, or the size of its metatype).

Rejected

SE-0123: Disallow coercion to optionals in operator arguments is Rejected. Team writes:

The majority of the feedback on this proposal was opposed to it.  While the goal of the proposal is laudable, SE-0121 eliminated the most surprising problem that this proposal aimed to address, and the rest of it (notably the ?? operator) can be addressed in other ways in the Swift 3 timeframe.  The core team recommends that the ?? case be handled by having the compiler produce a warning when the left side of the ?? operator is implicitly promoted to optional.  In the post-Swift 3 timeframe, we can investigate other broader options for solving this problem more generally (e.g. a new parameter attribute).

SE-0119: Remove access modifiers from extensions is Rejected. Team writes:

The majority of the feedback on this proposal was opposed to it, because it eliminated the useful ability to apply access control to a batch of methods and properties.

Withdrawn For Revision

SE-0126: Refactor Metatypes, repurpose T.self and Mirror withdrawn for revision. Team writes:

The authors of SE-0126 have requested that we withdraw the proposal. The idea will be revised, return back to the “pitch” phase, and when that converges, SE-0126 will be revised.

Returned for Revision

SE-0117: Default classes to be non-subclassable publicly returned for a second revision. Team writes:

As with the first round of discussion, the community generated a large number of emails, exploring the various aspects of the proposal.  While many community members agree with the thrust of the proposal, a number of people are concerned with the boilerplate being introduced by the proposal, among other issues.  The core team spent over two and a half hours discussing this topic from first principles, and has come up with a similar-but-different approach that should reduce the boilerplate, while still accomplishing the primary aims of the proposal.  John McCall will be revising the proposal today and we’ll restart a short discussion period about it tomorrow.

In Review

I apologize for the formatting. Swift-Ev no longer lets me cut and paste from schedule.md to add the summaries. I tried cutting and pasting the new format and it didn’t work very well.

SE-0130: Replace repeating Character and UnicodeScalar forms of String.init in review to 7/24. “This proposal suggest replacing String initializers taking Character or UnicodeScalar as a repeating value by a more general initializer that takes a String as a repeating value. This is done to avoid the ambiguities in the current String API, which can be only resolved by explicit casting. It is also proposed to remove one of the String.append APIs to match these changes.”

SE-0128: Change failable UnicodeScalar initializers to failable in review to 7/24 “This proposal aims to change some (ones that are failable) UnicodeScalar initializers from non-failable to failable. i.e. in case a UnicodeScalar can not be constructed, nil is returned. Currently, when one passes an invalid value to the failable UnicodeScalar UInt32 initializer the swift stdlib crashes the program by calling _precondition. This is undesirable if one construct a unicode scalar from unknown input.”

SE-0127: Cleaning up stdlib Pointer and Buffer Routines in review to 7/24. “Update the API to match new API guidelines and remove redundant identifiers.”

SE-0126: Refactor Metatypes, repurpose T.self and Mirror in review to 7/24. “This proposal wants to revise metatypes T.Type, repurpose public T.self notation to return a new Type<T> type instance rather than a metatype, merge SE-0101 into Type<T>, rename the global function from SE-0096 to match the changes of this proposal and finally rename current Mirror type to introduce a new (lazy) Mirror type.”

SE-0117: Allow distinguishing between public access and public overridability revision 3 in review to 7/25 ” A public member will only be usable by other modules, but not overridable. An open member will be both usable and overridable. Similarly, a public class will only be usable by other modules, but not subclassable. An open class will be both usable and subclassable. This spirit of this proposal is to allow one to distinguish these cases while keeping them at the same level of support: it does not adversely affect code that is open, nor does it dissuade one from using open in their APIs. In fact, with this proposal, open APIs are syntactically lighter-weight than public ones.”

SE-0125

to 7/22

Remove NonObjectiveCBase and isUniquelyReferenced

This replacement is as performant as the call to isUniquelyReferenced in cases where the compiler has static knowledge that the type of object is a native Swift class and dyamically has the same semantics for native swift classes. This change will remove surface API.

SE-0124

to 7/20

Int.init(ObjectIdentifier) and UInt.init(ObjectIdentifier) should have a bitPattern: label

This makes it clear at the use site that we interpret the value as a bit pattern

SE-0123

to 7/19

Disallow coercion to optionals in operator arguments

The convenience of coercing values to optionals is very nice in the context of normal function calls, but in the context of operators, it can lead to some strange and unexpected behavior.

SE-0122

to 7/24

Use colons for subscript declarations

Matches accessor declarations elsewhere in the language

SE-0121

to 7/19

Remove Optional Comparison Operators

Doesn’t work in current versions of Swift, because generics don’t support conditional conformances like extension Optional: Comparable where Wrapped: Comparable, so Optional is not actually Comparable.

SE-0120

to 7/19

Revise partition Method Signature

A more general partition algorithm using a unary (single-argument) predicate would be more flexible and generally useful.

SE-0119

to 7/19

Remove access modifiers from extensions

This proposal aims to fix access modifier inconsistency on extensions compared to other scope declarations types.

SE-0117

to 7/22

Default classes to be non-subclassable publicly

Not all classes make sense to subclass, and it takes real thought and design work to make a class subclassable well.

SE-0116

to 7/11

Import Objective-C id as Swift Any type

Objective-C’s id type is currently imported into Swift as AnyObject. This is the intuitive thing to do, but creates a growing tension between idiomatic Objective-C and Swift code.

SE-0101

to 7/19

Reconfiguring sizeof and related functions into a unified MemoryLayout struct

Discards the value-style standalone functions and combines the remaining items into a unified type.