Apple's Next Operating Systems: How and Why to Test Them Now

Now that Apple’s next operating systems are available via prerelease software programs, it’s time for you as an Apple admin to test them. This year, that means macOS Sonoma, iOS 17, iPadOS 17, tvOS 17, and (newly relevant to Apple admins) watchOS 10.

The reason is simple: You need to evaluate a new OS before deploying it widely, so you know how well (or if) it will get along with your existing technical infrastructure. 

But it’s one thing to know you need to test and another to get that testing done. Here are a few guidelines for ensuring that you'll be ready when those OSes ship this fall. For the purposes of this post, we'll focus on macOS, iOS, and iPadOS.

Downloading Apple Beta Software

The first step in testing is to actually acquire the software. Apple distributes beta versions of its operating systems through several channels, including its Beta Software Program, Apple Developer, and AppleSeed for IT programs. All three let users, developers, and IT professionals test the compatibility of new OS versions with their apps and environments; in return, Apple asks only that those users submit feedback.

But of those three options, only AppleSeed for IT is designed specifically for IT admins who need to test new versions of Apple beta software in managed environments. Participating in AppleSeed is the best way to set up your organization for success when the latest OSes are released.

AppleSeed for IT provides:

• Access to prerelease software: AppleSeed lets you test operating systems and select Apple apps in your business environment before they’re released.
• Detailed test plans and surveys: Apple supplies test plans—including regression and new-feature tests—to help you evaluate the compatibility of beta software with your existing tech infrastructure. 
• Dedicated review queue: When you submit feedback through AppleSeed, it’s funneled to a dedicated review queue to make sure bug submissions and enhancement requests get to the right people at Apple as soon as possible.
• Discussion forums: When you join AppleSeed, you gain access to its exclusive discussion forums. They’re a great place to post questions, help other participants, and discuss other IT topics. 
• Debug profiles: In addition to the debug profiles available on the Apple Developer website, AppleSeed also includes debug profiles for specific technologies where they would like to focus testing.

AppleSeed for IT requires that your organization is enrolled in Apple Business Manager or Apple School Manager. Anyone with a Managed Apple ID in your organization can participate (except for those with a Student role under Apple School Manager).

To learn more about AppleSeed for IT and to find out how to access these services and tools, go to beta.apple.com/it and sign in with your Managed Apple ID.

According to the terms and conditions, you should not share software you download from AppleSeed with anyone else—even within your organization. The best way to have others participate in testing is to have them sign in with their own Managed Apple IDs.

Installing Apple Beta Software

To install beta releases for macOS Sonoma, iOS 17, or iPadOS 17, you’ll need a Managed Apple ID that’s enrolled in the AppleSeed beta program; Apple IDs registered with the Apple Developer Program or with the Beta Software Program will also work. Keep in mind that the public betas are released less often than those in AppleSeed.

Once you have the appropriate ID, you'll open Settings (on iOS or iPadOS) or System Settings (on macOS), navigate to General > Software Update > Beta Updates, and sign in. That done, you can choose the Developer Beta for the upcoming operating system for the device. This will allow you to install the latest beta over the air. Subsequent betas will be available on the device when Apple releases them.

If you want to want to test upgrading or other scenarios that benefit from a more “natural” environment, you can enroll a device running the current operating system. But if you want to test on a completely clean system, IPSWs are available for download from AppleSeed for IT, for all platforms except watchOS. You can use Apple Configurator for Mac to install those system images on target devices (which must be erased). Just remember: Whichever operating system you’re testing, don't install it on production devices. 

Planning Your Testing Program

What’s the best way to test a prerelease OS? The specific answers obviously depend entirely on your specific circumstances: the kinds and quantity of Apple hardware you manage, the apps and services your organization uses, how much time you have, and so on. But here are some general guidelines to think through.

1) Use Multiple Platforms

Before you do any testing, you need to settle on a test platform—or, more accurately, platforms. Our recommendation: Do it in stages.

Start by installing the new OS on a clean, wiped Mac, iPhone, or iPad. These systems should be dedicated to testing, not actually in production use. As much as possible, configure the test platform as you would any new production computer or device, with your standard Wi-Fi and other settings, then run it through your suite of tests.

That done, try upgrading a test system that’s currently running the last generation OS: from macOS Ventura to macOS Sonoma on the Mac, from iOS or iPadOS 16 to 17 on iPhone or iPad. Once again, configure these devices as you would a production system, then run them through your test suite.

Many admins use virtual machines for testing. VMs can indeed be a boon but be warned: Because they don’t always replicate physical machines perfectly, VMs can have limitations as testing platforms. Admins can even take a page from Agile and DevOps by leveraging the Virtualization framework to automate some of their testing. Virtual machines can be a strong component of a testing plan, but shouldn’t be the only one.

2) Start with the OS Itself

You want to take a tiered approach for each platform, starting with the new operating system itself. Does it install? Does it work? Can you connect to local networks (wired and wireless) and the internet? Does Bluetooth work?

If the new OS doesn’t work on a clean Mac, there might be a known issue on the Apple side. Check the release notes on AppleSeed for IT to see if there are known issues that might explain the problem and provide feedback (see below).

3) Test Your Apps

Once you are comfortable with the fundamentals of the operating system, turn next to applications. What are your five most critical user-facing tools? Focus on productivity apps—Microsoft Office or the Adobe Creative suite, for example—on which your organization relies. Avoid email and calendaring for now, as they depend on the back-end infrastructure that you’ll test next. 

Install these apps on your test hardware and run them through their paces. Does the app install and launch under the new OS? Does it work as you expect? If either of those answers is no, contact the developer of the app in question and let them know. File a report with Apple, as well.

4) Check Your Infrastructure

Next, test the more complex apps and services that your organization relies on. 

Can the test system connect successfully to your email services? If you use Exchange, for example, does it function as expected? What about other cloud services, such as your VPN and your identity provider? Do your security tools work? Security apps, in particular, frequently break with new operating systems.

One special component of that infrastructure that you should test: Your MDM solution. Does the test device with the beta OS enroll correctly? Try deploying a current configuration. Does it all work? Next, can you use your MDM to deploy the beta OS to the test hardware? Again, try it first on a freshly wiped system, then one that you’re upgrading from a previous OS.

Generally, whatever worked with the previous operating system should work with the upcoming one, unless a particular feature has been intentionally removed. If something doesn't work that should, you should report the unexpected behavior to Apple and your MDM vendor. Most such vendors don’t make functionality that is new to the upcoming OSes available until Apple officially ships those operating systems. That can make it difficult to test new features. But remember that, with most MDMs, you can craft your own profile and have the MDM distribute it for you.

Beta Software Test Plans from Apple

One great resource for test plans is Apple itself. The company supplies sample plans as part of its AppleSeed for IT program. On the AppleSeed site, sign in, click the Resources tab, then scroll to Test Plans & Documentation. These plans cover a wide range of topics, including Microsoft Exchange, Platform SSO, and more. These often represent areas where Apple made a lot of changes so they can confirm that everything is working before release. They're also incredibly thorough. New test plans can appear later in the beta cycle, so it is worth checking periodically.

As an example, Apple provides the macOS Testing Template for testing macOS. It covers the testing deployment and management, and Infrastructure and Network. It also gives you space to collect the testing status of various kinds of apps and provides examples of a variety of specific test cases.

You will likely find items in Apple’s plans you’ve never even considered testing. Even if you don’t need to test the specific scenarios or conditions covered by Apple’s test plans, they’re excellent templates to use in planning tests of your own.

How Much Testing Should You Do?

If your time or resources are limited, keep your plan simple: Test the new OS to see whether it installs correctly on your hardware. If you have a mix of Intel-based and Apple silicon Mac computers, test the new OS on both. See if you can create a new user and enroll in your MDM solution. If you can take care of those basics, that might be enough for now.

We can’t stress enough how important it is to do as much testing as you can before you roll out a new OS to your users. But if you absolutely don’t have the time or resources for even the most minimal testing program, the fallback is to wait for the OS to be released to the public. When it is, install it on one machine, and see what (if anything) breaks. In the meantime, keep an eye on public forums (such as the MacAdmins Slack channel) to see what your peers are saying. 

If you can test more comprehensively, recognize that feedback provided to Apple and app vendors earlier in the beta cycle is more valuable than feedback provided later. It gives Apple and software vendors more time to diagnose and address problems, making it more likely those problems won’t be present in the shipping operating system or apps come release in the fall.

Providing Feedback

In light of this, while your main goal of prerelease testing likely is to check the forthcoming operating systems against your existing hardware and software, there’s another crucial goal: Providing feedback to Apple about the new OSes. For that, your best bet is AppleSeed for IT. 

When to submit feedback? Whenever an app or service you’ve used in the past doesn’t work or behaves oddly—crashing, hanging, or being so slow as to be unusable—under the new OS.

You submit that feedback to Apple through the Feedback Assistant app that’s built into the prerelease OSes. Sign in to Feedback Assistant using the same Managed Apple ID you use to sign in to AppleSeed. File feedback for your organization (rather than Personal) to ensure it gets routed to the AppleSeed for IT triage team. You should be prepared to submit device logs along with any feedback you provide. Along with your description of the problem and the time you encountered it, this gives Apple’s software engineers a starting point for diagnosing the problem.

When you file feedback, it's important to make it useful. Don't just say, "It doesn't work". The more information you can provide, the more likely your bug will get fixed fast and won’t require a lot of follow-up. As a start, include a detailed, descriptive title when filing the report, following the rubrics “[[service or app]] [[is doing this weird thing]]” or “[[can’t do this]] in [[service or app]].” Screenshots are also really helpful. For more, see Apple’s article on how to file good bug reports.

If the problem you’ve identified involves third-party software, make sure to use that vendor’s feedback mechanism as well. It can be difficult at this stage to tell whether a problem is with the operating system or the app, so you’re better off filing a report with both Apple and the ISV.

As mentioned above, if you file feedback using your organization credentials, it will be routed to a special AppleSeed for IT queue. That means your bug submissions and enhancement requests will go to people at Apple who can actually do something about them. 

If you do submit an issue, note the Feedback ID number you receive; it will be used in any follow-up between you and Apple. You can review the feedback you’ve filed or drafted in Feedback Assistant and on the AppleSeed website. Apple will often ask for additional information. These conversations will happen in Feedback Assistant. Your prompt response to their questions makes it more likely that issues can be addressed before release.

One thing to keep in mind when considering whether to file feedback is that when you do so, you’re helping yourself and the community of Apple admins and users. Multiple reports of a problem may help Apple prioritize one issue over another. You may find an issue that other admins didn’t, but which will affect them later. Apple has these pre-release periods both to give customers time to react and adapt to changes and to ensure software quality. Your feedback is how they can do that. 

One final note: While you should be downloading and checking prerelease software in a dedicated testing environment, you probably don't want your users to do the same thing on their production devices. Your device management solution may be able to help with that. Here, for example, is how to restrict access to beta OS releases in Kandji.

Editor's Note: This post was originally published July 19, 2022. It was substantially updated July 13, 2023.

About Kandji

By testing betas and leveraging an MDM solution, you can ensure your organization’s transition to the new operating systems is smooth. Kandji supports deploying beta profiles, so you can remotely install those prerelease OSes on devices in a testing group. With powerful features like zero-touch deployment, one-click compliance, and offline remediation, Kandji has everything you need to enroll, configure, and secure your devices.