Server Driven UI

This post is based on a transcription of a talk I gave at CocoaHeadsNL in July 2020. The talk, including a live Q&A afterwards, can be seen here:

Over the years, I’ve build many different types of apps. However in one aspect they’ve all been very similar; In the client/server architecture, the server sends domain objects to the client (encoded in JSON), and the client renders these domain objects to some pretty UI.

Server Driven UI is different. The server does not send domain objects, with the client having to decide how to render those. Instead, the server decides what and how to render, and just sends instructions to the client. You know, kinda like HTML…

(Honestly, it’s not HTML… But it sort of is. But really, it isn’t)

Continue reading “Server Driven UI”

@CloudStorage property wrapper

I wrote a Swift library to help with prototyping (haven’t used it yet in production apps).

CloudStorage is property wrapper that will save values in iCloud key-value storage. These values are persisted across app restarts and will sync between different devices that are signed into the same iCloud account. This property wrapper is similar to AppStorage and SceneStorage, two new types Apple introduced with iOS 14.

The basic API is the same as for AppStorage; add @CloudStorage("someName") in front of a property, to save the value to iCloud key-value storage.

Usage

Make sure you enable the “key-value storage” service in the iCloud capability. See Apple’s documenation.


@CloudStorage("readyForAction") var readyForAction: Bool = false
@CloudStorage("numberOfItems") var numberOfItems: Int = 0
@CloudStorage("orientation") var orientation: String?

See also the example app in the repository.

For what should this be used?

The same caveats apply as with key-value storage itself:

Key-value storage is for discrete values such as preferences, settings, and simple app state.
Use iCloud key-value storage for small amounts of data: stocks or weather information, locations, bookmarks, a recent documents list, settings and preferences, and simple game state. Every app submitted to the App Store or Mac App Store should take advantage of key-value storage.

From Apple’s documentation on choosing the proper iCloud Storage API.

In general, key-value storage is not meant as a general purpose syncing service. If you need any advanced capabilities to prevent data loss, consider using CloudKit instead.

Check out the CloudStorage repository.

My wish for watchOS: Dutch-style Cycling support

Screenshot of Outdoor Cycle activity on Apple Watch

Currently, watchOS has a workout called “Outdoor Cycle”, the icon shows a person on a racing bicycle. But that is not what cycling means, for millions of people who use a utility bicycle for daily transportation.

Instead of specific activity, that some people do, some of the time. Cycling can also be a boring part of everyday life, that everyone does, all of the time.

How cycling in The Netherlands is different

Here I’m going to make some assumptions and generalisations about places I don’t live… When I think of cycling in some other places, like say California, I think of something like this:

Here we see people, hunched over on racing or road bikes. They go fast, wear special protective clothing and are are really working out. More importantly, culturally these people are doing something abnormal, in the sense that cycling is not the norm, most people don’t cycle regularly.

In contrast, this is a typical Dutch view of cycling:

In The Netherlands, cycling is a common mode of transport. 27% of all trips are made by bicycle. In Amsterdam, a third of morning commutes are made by bicycle, making it the most popular form of transport (above walking and cars). At first glance you can see the differences: people sit upright, drive slowly, and they are wearing ordinary clothing. The bicycles themselves are also different; They are relatively cheap, have a kickstand, a chain guard and angled-back handlebars. They also include one or two luggage carriers. 

Side note; It’s not that the other type of bicycle doesn’t exist in The Netherlands. We too have people who (in the weekends), put on a helmet, dress up in lycra, and take out their other sporting bicycle. To do competitive racing, or mountain biking in the woods. But those are recreational sports, not day-to-day transport.

My which for Apple Watch: Support for Utility Cycling

I would like it if watchOS 7 added an additional option. To support this normal, everyday, utility cycling. It seems a bit weird to call this a “workout”, but hey “outdoor walking” is also in there, so what the heck. I even designed an icon for this:

Screenshot of Apple Watch Workouts app with hypothetical Utility Cycle activity

In a similar way to Outdoor walking, this activity should be automatically detected and activated. This could be used to more accurately track activity when commuting to work, taking the kids to school or grocery shopping. And while you’re at it Apple, please also add the cycling modality to Apple Maps!


PS: For those unfamiliar, here’s a little mood impression of what cycling in The Netherlands looks like:

(Also, do check out the rest of the rest of BicycleDutch channel on YouTube. It’s great!)

Every two years, Apple increases software support by a year

Over the last 12 years, a pattern has emerged: Every two years Apple has adds an extra year of iOS support to its System-on-chips (SoCs).

The original iPhone and iPhone 3G both had three years of software updates. The iPhone 3GS and iPhone 4 had four years of updates, and so on.

This diagram visualises that pattern:
iOS versions per SoC. 2007 and 2008: 3 years. 2009 and A4: 4 years. A5 and A6: 5 years. A7 and A8: 6 years. A9 and A10: 7 years? A11 and A12: 8 years?

The coloured lines indicate iOS support a specific SoC. The grey line indicates when devices with this SoC were sold.

Note that this diagram specifically plots SoCs, and not iPhone releases. The same A-series SoCs are used across iPhones, iPads and iPod touches. I’ve merged the releases of several of these devices to create this diagram.

Should this trend continue in the future, the A8 will be supported for 6 years. The A9/A10 for 7 years, and the A11/A12 for 8 years.

That means the iPhone XS released in September 2018, which runs the A12 Bionic, would get updates until 2025, with iOS 19.

The iPhone 6 exception

Again, the above diagram refers to SoCs in general, and not iPhones, because if we plot that, there’s an exception for the iPhone 6.
iOS versions per iPhone. iPhone 6 is only supported for 5 years, but it's SoC is supported for 6 years.

The coloured lines indicate support iOS support for an iPhone. The grey line indicates when the iPhone sold.

The iPhone 6 was released in September 2014 and was for sale until March 2017.

In September 2019, when iOS 13 comes out, the iPhone 6 will no longer be supported. This is unfortunate because a lot of users are still using iPhone 6 devices. At Q42 we see it is the 6th most popular iPhone with 7% usage, more popular than the iPhone SE or iPhone XR.

Although support for the iPhone 6 (and iPod touch 6th gen) is being dropped, the iPad Air 2 and iPad mini 4 of the same generation are still supported. The iPad mini 4 uses the same A8 SoC as the iPhone 6.

A possible explanation for why it is being dropped might be the size of the RAM. iPhone 6 uses an A8 SoC, but it only has 1GB of RAM. The iPads both have 2GB of RAM.

Every device supported by iOS 13, has at least 2GB of RAM.

The future?

It is impossible to predict the future, but I don’t think this “RAM filter” will occur again soon.

The next obvious step would be some future version of iOS that only supports 3GB RAM. But that would cut product lines in half. iPhone 7 and 8 both have 2GB or RAM, but the iPhone 7 Plus and 8 Plus use 3 GB.

The step after that is 4GB as a minimum, but that includes some very recent devices. The iPhone XR (September 2018) and iPad Air 3 (March 2019) both ship with 3GB of RAM.

My assumption is that, for the coming years, device deprecations will again be solely based on SoC.

So…

Will this trend of adding a support year continue indefinitely? If not, when will it stop? Will there be another iPhone 6-style hiccup? Who knows!

I can’t wait for WWDC 2020, to see what iOS 14 brings!


All data about SoCs and release years are taken from Wikipedia. This chart lists all iOS-like devices from the past 12 years.

Hacking “Marzipan” – Running iOS apps on Mac [talk]

A recording from my talk at the Do iOS conference last month.

At WWDC 2018 Apple announced that in the future it will become possible to run iOS apps on macOS. Rumours about this first appeared in December 2017 and was believed to be codenamed Marzipan. Although there is no official API or support for it yet in this talk Tom Lokhorst shows how he experimented with Marzipan and what he learned.

The example project used in the talk is available on GitHub: MarzipanDemoApp.

Machine Learning + AR at the Rijksmuseum

For the past four months Daniello, an intern at Q42, has worked on training a machine learning model to recognise artworks in the Rijksmuseum. Once recognised, the paintings are located in the AR camera feed and augmented using ARKit.

This week at WWDC, Apple announced CreateML and ARKit 2. Those two solutions combined may make this process a lot easier, but nevertheless, it’s really cool to see what Daniello build:

Art recogniser Rijksmuseum from Q42.

Rijksmuseum ArtViewer [talk]

A recording from my talk at CocoaHeadsNL last month

Tom Lokhorst from Q42 talks about how they developed the ArtViewer for the Rijksmuseum app. This viewer is a highly optimised image viewer to display very large images. It efficiently uses caching and tiling to save memory and bandwidth.

Rijksmuseum ArtViewer, Tom Lokhorst from CocoaHeadsNL.

See the Rijksmuseum app in the App Store.

Leaky abstractions in Swift with DispatchQueue

At Q42 we have some apps with very occasional weird multi-threading issues. After a bunch of debugging Mathijs Kadijk and I figured out it had something to do with DispatchSpecificKey. This post details what we found out.

Pop quiz! What do you think this prints?


while true {
  let key = DispatchSpecificKey<Int>()

  if let value = DispatchQueue.main.getSpecific(key: key) {
    print("ERR \(value)")
    exit(1)
  }
  else {
    DispatchQueue.main.setSpecific(key: key, value: 42)
    print("OK")
    sleep(1)
  }
}

Answer: This does not print a infinite stream of OKs every second, as I would have thought. Instead, it stops after three iterations of the loop:


OK
OK
ERR 42
Program ended with exit code: 1

The output changes a bit each time I run the program. Sometimes it makes it to three or four OKs, but often it only prints one OK and then stops with an ERR.

My question after seeing this behaviour: Wut? How can two distinct keys result in the same value?!?

Wrappers around C APIs

It turns out these (woefully underdocumented) Swift APIs are wrappers around C APIs. The code for this is all open source, see Queue.swift. The underlying C APIs are better documented and give some insight into how this all works internally:

The documentation for dispatch_queue_set_specific explains how the key is just a pointer to something, it doesn’t really matter what it’s pointing to.

Keys are only compared as pointers and are never dereferenced. Thus, you can use a pointer to a static variable for a specific subsystem or any other value that allows you to identify the value uniquely.

For this reason, the Swift implementation of DispatchSpecificKey is very simple:


public final class DispatchSpecificKey<T> {
	public init() {}
}

The initialised object is only used to get a pointer value that can be passed to the dispatch_queue_set_specific function. This pointer should of course be unique (hint: it is not).

Debugging the weird behaviour

Suspecting this behaviour has something to do with the pointer, lets add a print statement to see what the pointer value is:


let p = Unmanaged.passUnretained(key).toOpaque()
print("Pointer: \(p)")

This resulted in the following output:


Pointer: 0x000000010120b590
OK
Pointer: 0x000000010104ab00
OK
Pointer: 0x000000010120b590
ERR 42
Program ended with exit code: 1

Finally, this the explains the behaviour we’re seeing; Different instances of DispatchSpecificKey share the same pointer!

Presumably ARC cleans up the key variable after we’re no longer using it and in a next iteration of the loop, the same memory location is reused again to store a new instance of DispatchSpecificKey.

Possible fixes for the bug

There are different solutions to fix this unexpected behaviour. One is to simply keep an array of each DispatchSpecificKey that gets created, that way no two DispatchSpecificKeys get assigned to the same memory location.

In my own code, I happened to have the key be a member of an object. The solution was to add a deinitializer to the object:


class MyObject {
  let key = DispatchSpecificKey<Int>()

  deinit {
    DispatchQueue.main.setSpecific(key: key, value: nil)
  }
}

A more general solution would be if DispatchSpecificKey were to be updated to clean up after itself:


public final class DispatchSpecificKey<T> {
  public init() {}

  // Is it OK to keep strong references to queues?
  internal var queues: [DispatchQueue] = []

  deinit {
    for queue in queues {
      queue.setSpecific(key: self, value: nil)
    }
  }
}

Closing thoughts

It took us a lot of debugging to finally narrow down the source of our bug to this reusing of the same memory address by two distinct objects. Normally in Swift code this wouldn’t be a problem, but because the underlying C API uses just pointers to compare for equality, this suddenly matters.

For the C code, it is arguable that the programmer should be responsible for cleaning up after they’re done with a C “dispatch specific key”. They should call dispatch_queue_set_specific with a nil value themselves.

But for the Swift code; The DispatchSpecificKey class really implies programmers shouldn’t have to know about the pointer internals of the C API. So in my opinion that class should have the cleanup code build-in.

Xcode 9 + iOS 10 – Preserve Superview Margins bug

Update 2017-08-25: This bug appears to be fixed in Xcode 9, beta 6.


On Xcode 9, when compiling to iOS 10; views nested in root view with “Preserve Superview Margins” enabled, have an incorrect margin of 8px.

iOS Simulator showing 8px margin

I’ve filed a radar with Apple:

ID: 33992313

Summary

Xcode 9 (beta 5) generates incorrect margins when building for iOS 10 with “Preserves Superview Margins” enabled in Interface Builder.
The same code works correct on Xcode 9 + iOS 11, and also correct on Xcode 8 + iOS 10.

Steps to Reproduce

In Xcode 9, beta 5 / iOS 10:

  • Create a “Wrapper View” in interface builder directly under root view of view controller
  • Pin edges to superview edges on Wrapper View
  • Enable “Preserve Superview Margins” on Wrapper View
  • Add subview to Wrapper View that is pinned to the Wrapper View layout margins

Expected Results

Left and right margins should be 16 or 20, depending on device.

Observed Results

Left and right margins are always 8.

Notes

Does not occur when building with Xcode 8 (for iOS 10) or Xcode 9 (for iOS 11)

Side note:
Setting explicit layout margins instead of using “Preserve Superview Margins” only works after rotating screen.
But this is probably an unrelated iOS 10 bug, as this also happens in Xcode 8.