Apple will be changing up its testing strategy after a swarm of bugs marred the latest iPhone and iPad operating systems. The new approach calls for Apple’s development teams to ensure that test versions, known as “daily builds,” of future software updates disable unfinished or buggy features by default. Testers will then have the option to selectively enable those features, via a new internal process and settings menu dubbed Flags, allowing them to isolate the impact of each individual addition on the system.
When the company’s iOS 13 was released alongside the iPhone 11 in September, iPhone owners and app developers were confronted with a series of software glitches. Apps crashed or launched slowly. Cellular signal was inconsistent. There were user interface errors in apps like Messages, system-wide search issues and problems loading emails. Some new features, such as sharing file folders over iCloud and streaming music to multiple sets of AirPods, were either delayed or are still missing. This amounted to one of the most troubled and unpolished operating system updates in Apple’s history.
The issues show how complex iPhones have become and how easily users can be disappointed by a company known for the smooth integration of hardware and software. Annual software updates timed for release with the latest iPhones are a critical way for Apple to add new capabilities and keep users from defecting to archrival Android. Refreshed operating systems also give developers more tools for app creation, catalyzing more revenue for Apple from its App Store.
The new development process will help early internal iOS versions to be more usable, or “livable,” in Apple parlance. Prior to iOS 14’s development, some teams would add features every day that weren’t fully tested, while other teams would contribute changes weekly.
Apple measures and ranks the quality of its software using a scale of 1 to 100 that’s based on what’s known internally as a “white glove” test. Buggy releases might get a score in the low 60s whereas more stable software would be above 80. iOS 13 scored lower on that scale than the more polished iOS 12 that preceded it. Apple teams also assign green, yellow and red color codes to features to indicate their quality during development. A priority scale of 0 through 5, with 0 being a critical issue and 5 being minor, is used to determine the gravity of individual bugs.
The new strategy is already being applied to the development of iOS 14, codenamed “Azul” internally, ahead of its debut next year. Apple has also considered delaying some iOS 14 features until 2021 – in an update called “Azul +1” internally that will likely become known as iOS 15 externally – to give the company more time to focus on performance. Still, iOS 14 is expected to rival iOS 13 in the breadth of its new capabilities, the people familiar with Apple’s plans said.
The testing shift will apply to all of Apple’s operating systems, including iPadOS, watchOS, macOS and tvOS. The latest Mac computer operating system, macOS Catalina, has also manifested bugs such as incompatibility with many apps and missing messages in Mail. Some HomePod speakers, which run an iOS-based operating system, stopped working after a recent iOS 13 update, leading Apple to temporarily pull the upgrade. The latest Apple Watch and Apple TV updates, on the other hand, have gone more smoothly.
Last year, Apple delayed several iOS 12 features – including redesigns for CarPlay and the iPad home screen – specifically so it could focus on reliability and performance.
About a month before Apple’s 2019 Worldwide Developers Conference in June, the company’s software engineers started to realize that iOS 13, then known internally as “Yukon,” wasn’t performing as well as previous versions. Some people who worked on the project said development was a “mess.”
By August, realizing that the initial iOS 13.0 set to ship with new iPhones a few weeks later wouldn’t hit quality standards, Apple engineers decided to mostly abandon that work and focus on improving iOS 13.1, the first update. Apple privately considered iOS 13.1 the “actual public release” with a quality level matching iOS 12. The company expected only die-hard Apple fans to load iOS 13.0 onto their phones.
The timing of the iOS 13.1 update was moved up by a week to Sept. 24, compressing the time that iOS 13.0 was Apple’s flagship OS release. New iPhones are so tightly integrated with Apple software that it would have been technically impossible to launch the iPhone 11 with iOS 12, and since 13.1 wasn’t ready in time, Apple’s only choice was to ship with 13.0 and update everyone to 13.1 as quickly as it could.
While the iOS 13 issues did upset iPhone owners, they still updated fairly quickly. As of mid-October, half of all Apple device users were running a version of iOS 13, according to Apple. That upgrade pace is still far ahead of Google’s Android.
Once iOS 13.1 was released, Apple’s software engineering division pivoted to iOS 13.2 with a quality goal of being better than iOS 12. This update has had fewer complaints than its predecessors in the iOS 13 family but did introduce a short-lived bug around apps closing in the background when they shouldn’t.