Swift Testing: My First Impressions

Introduction Recently, I posted here an Introduction to Swift Testing: Apple's New Testing Framework, in which I talked about the framework features and specially about the differences compared to XCTest (most improvements). Since then, I've been testing it a lot — no pun intended — in personal projects and I could create personal impressions about Swift Testing. Unfortunately, I didn't have the opportunity to use it in commercial projects yet, but with I have so far I think it's worth a post about my findings. If you didn't read the introduction post yet, I highly recommend you reading it first, since here I'll skip the basics and consider you already know how Swift Testing works. Testing Asynchronous Code Something that I didn't cover in my first post about Swift Testing is testing asynchronous code. Testing code with with async/await is so easy as testing synchronous code. You just need make your testing function async, and you are able to use await wherever it's needed: @Test("Test that calling the backend endpoint fetches data") func testBackend() async { let apiClient = ApiClient() let data = await apiClient.fetchDataFromBackend() #expect(!data.isEmpty) } This is cool because it encourages you to update your logic to the async/await paradigm. But if you use another approach to handle asynchronous execution, you'll have some more work to test it with Swift Testing. While in XCTest we had the expectations to test asynchronous code, Swift Testing doesn't provide any specific tool/structure to do the same. What you need to do is to use the native Swift withCheckedContinuation or withCheckedThrowingContinuation functions. Let's see an example with a function with completion handler: Function to be tested: func auth(with credentials: Credentials, completion: @escaping (Result) -> Void) -> URLSessionTask { // ... } Test: @Test("Test that calling the auth endpoint stores the JWT") func testAuth() async throws { let apiClient = DummyJSONAPIClient() let credentials = Credentials(username: "emilys", password: "emilyspass") await withCheckedContinuation { continuation in let _ = apiClient.auth(with: credentials) { result in if case .success = result { #expect(apiClient.jwt != nil) } else { Issue.record("Authentication Failed") } continuation.resume() } } } This code is copied from my last post about Creating a Network Layer with URLSession in Swift. Check it out! Here we just mark our testing function with async and use withCheckedContinuation to "convert" the completion handler approach in async/await. You could do the same thing to test code with Promises, reactive frameworks (as Combine and/or RxSwift) or any other asynchronous context. I find this approach very cool and compliant with the "idea" of Swift Testing being a very swifty framework. You don't use any framework-specific structure to handle asynchronous execution, you just do it the way the language provides it to do. Explicitly Failing Tests In the previous post, I talked about how Swift Testing replaces all the XCTAssert functions from XCTest with the single #expect macro. Instead of having multiple functions to each kind of comparison we need to do, the #expect macro receives a boolean, allowing the developer to pass the desired condition to it. But something that I missed in the last post is when you need to purposely make the test fail. This happens usually with types that cannot be used in conditions, as Results and enums: apiClient.auth(with: credentials) { result in #expect(result == .success) // ❌ Cannot convert value of type 'Result' to expected argument type 'DispatchTimeoutResult' } In that case, you need to use other strategies to check the result value, and then decide if the test should pass or fail: switch(result) { case .success(): // ✅ Pass the test case .failure(_): // ❌ Fail the test } For passing the test, nothing really needs to be done. A test without any #expect ran is considered a passed one. But for failing, you'll need to add an explicit instruction. The most obvious one would be #expect(false), and it works, but Xcode will give you an warning saying that the expectation will always fail: case .failure(_): #expect(false) // ⚠️ '#expect(_:_:)' will always fail here; use 'Bool(false)' to silence this warning (from macro 'expect') The warning itself already gives you the solution for that: using Bool(false) instead of false: case .failure(_): #expect(Bool(false)) The warning is now gone, however there's also another way to make the test fail, that in my opinion is better and more legible: using the static Issue.record(_) function: case .failure(_): Issue.record("Authentication Failed") Why I think it's better: it's mandatory adding a comment. This comm

Apr 4, 2025 - 15:39

Introduction

Recently, I posted here an Introduction to Swift Testing: Apple's New Testing Framework, in which I talked about the framework features and specially about the differences compared to XCTest (most improvements). Since then, I've been testing it a lot — no pun intended — in personal projects and I could create personal impressions about Swift Testing. Unfortunately, I didn't have the opportunity to use it in commercial projects yet, but with I have so far I think it's worth a post about my findings.
If you didn't read the introduction post yet, I highly recommend you reading it first, since here I'll skip the basics and consider you already know how Swift Testing works.

Testing Asynchronous Code

Something that I didn't cover in my first post about Swift Testing is testing asynchronous code. Testing code with with async/await is so easy as testing synchronous code. You just need make your testing function async, and you are able to use await wherever it's needed:

@Test("Test that calling the backend endpoint fetches data") 
func testBackend() async {
    let apiClient = ApiClient()
    let data = await apiClient.fetchDataFromBackend()
    #expect(!data.isEmpty)
}

This is cool because it encourages you to update your logic to the async/await paradigm. But if you use another approach to handle asynchronous execution, you'll have some more work to test it with Swift Testing.

While in XCTest we had the expectations to test asynchronous code, Swift Testing doesn't provide any specific tool/structure to do the same. What you need to do is to use the native Swift withCheckedContinuation or withCheckedThrowingContinuation functions. Let's see an example with a function with completion handler:

Function to be tested:

func auth(with credentials: Credentials, completion: @escaping (Result<Void, NetworkError>) -> Void) -> URLSessionTask {
    // ...
}

Test:

@Test("Test that calling the auth endpoint stores the JWT") 
func testAuth() async throws {
    let apiClient = DummyJSONAPIClient()
    let credentials = Credentials(username: "emilys", password: "emilyspass")

    await withCheckedContinuation { continuation in
        let _ = apiClient.auth(with: credentials) { result in
            if case .success = result {
                #expect(apiClient.jwt != nil)
            } else {
                Issue.record("Authentication Failed")
            }

            continuation.resume()
        }
    }
}

This code is copied from my last post about Creating a Network Layer with URLSession in Swift. Check it out!

Here we just mark our testing function with async and use withCheckedContinuation to "convert" the completion handler approach in async/await. You could do the same thing to test code with Promises, reactive frameworks (as Combine and/or RxSwift) or any other asynchronous context.

I find this approach very cool and compliant with the "idea" of Swift Testing being a very swifty framework. You don't use any framework-specific structure to handle asynchronous execution, you just do it the way the language provides it to do.

Explicitly Failing Tests

In the previous post, I talked about how Swift Testing replaces all the XCTAssert functions from XCTest with the single #expect macro. Instead of having multiple functions to each kind of comparison we need to do, the #expect macro receives a boolean, allowing the developer to pass the desired condition to it. But something that I missed in the last post is when you need to purposely make the test fail. This happens usually with types that cannot be used in conditions, as Results and enums:

apiClient.auth(with: credentials) { result in
    #expect(result == .success)
    // ❌ Cannot convert value of type 'Result' to expected argument type 'DispatchTimeoutResult'
}

In that case, you need to use other strategies to check the result value, and then decide if the test should pass or fail:

switch(result) {
case .success():
    // ✅ Pass the test
case .failure(_):
    // ❌ Fail the test
}

For passing the test, nothing really needs to be done. A test without any #expect ran is considered a passed one. But for failing, you'll need to add an explicit instruction. The most obvious one would be #expect(false), and it works, but Xcode will give you an warning saying that the expectation will always fail:

case .failure(_):
    #expect(false)
    // ⚠️ '#expect(_:_:)' will always fail here; use 'Bool(false)' to silence this warning (from macro 'expect')

The warning itself already gives you the solution for that: using Bool(false) instead of false:

case .failure(_):
    #expect(Bool(false))

The warning is now gone, however there's also another way to make the test fail, that in my opinion is better and more legible: using the static Issue.record(_) function:

case .failure(_):
    Issue.record("Authentication Failed")

Why I think it's better: it's mandatory adding a comment. This comment will appear in the test logs, helping you identify why a test is failing in a CI/CD environment, for example. Also, it's easier for other people understand why you are failing that test. Also, every expectation that fails calls the Issue.record function under the hood. But when using a #expect(Bool(false)), the message passed will be a generic one, generated by the framework.

Known Issue About Recording Tests Issues

At the moment I write this post, there's an open issue about issues recording in detached tasks. It's reported in Swift Testing Github Issues, but actually it seems to be a problem with Xcode, and not with the framework itself.
The issue is that when a test fail in a detached task (for example in the async response of a network call), Swift Testing crashes. That happens because in this async context, Swift Testing isn't able to detect for which test that error recording is.
This problem has relation with a very interesting Swift Testing feature: parallel execution. Swift Testing by default uses Swift Concurrency to execute tests in parallel, optimizing performance. The problem is that, since multiple tests can be running at same time, in a detached task is hard to know which test created that task.

Build Time Performance

That's something that I didn't really test, but I read in many places about: Swift Testing seems to have a worse perfomance in the build process compared to XCTest.
In June of 2024, people were reporting that build time could be 24x slower compared to XCTest.
There are some reasons for that, but the main one is that Swift Testing highly rely on Macros. If you don't know, Macros are "shortcuts" to bigger chunks of code. Examples of it in Swift Testing are @Test, #expect and #require. This process of converting the macro identifier into its corresponding code is called expansion. And expanding macros has a cost in build time.

Good News

The good news are that this PR and this PR from July of 2024 introduced a performance optimization to the build process, which made it 2.6x times faster, being more close to XCTest performance. We'll probably see more optimizations like these in future.

Overall Opinion

Swift Testing seems to be very promising, and I see Apple making it the official way of writing Unit Tests in near future. However you should be aware that it's a new technology, and some problems needs to be addressed. Also, it's an open-source framework managed by the Swift team, so some integrations with Xcode can be faced in short-term.