Apple's Next Operating Systems: How and Why to Test Them Now

We say it every year at this time: Now that Apple’s Apple’s beta software programs for its next operating systems are open (this year, that includes macOS Sequoia, iOS and iPadOS 18, watchOS 11, and visionOS 2), it’s time for you as an Apple admin to begin testing them. 

The reason is simple: You need to evaluate a new OS before deploying it widely, so you know how well your existing technical infrastructure will get along with it. 

But it’s one thing to know you need to test and another to get that testing done. Here are a few guidelines to ensure that you'll be ready when those OSes ship in a couple of months.

Downloading Prerelease Operating Systems

The first step in testing is to actually acquire the software. Apple distributes beta versions of its operating systems through multiple channels, including its Beta Software Program, Apple Developer, and AppleSeed for IT programs. All three let users, developers, and IT professionals test the compatibility of new OS versions with their apps and environments. In return, Apple asks only that those users submit feedback.

Of those three options, only AppleSeed for IT is specifically designed for IT admins who need to test new versions of Apple beta software in managed environments. Participating in AppleSeed is the best way to prepare your organization for success when the latest OSes are released.

AppleSeed for IT provides:

• Access to prerelease software: AppleSeed lets you test operating systems and some Apple apps in your business environment before they’re released.
• Detailed test plans and surveys: Apple supplies test plans—including regression and new-feature tests—to help you evaluate the compatibility of the software with your current environment. 
• Dedicated review queue: When you submit feedback through AppleSeed, it’s funneled to a dedicated review queue to ensure that bug submissions and enhancement requests reach the right people at Apple as soon as possible.
• Discussion forums: When you join AppleSeed, you gain access to its exclusive discussion forums. These are great places to post questions, help other participants, and discuss other IT topics. 
• Debug profiles: In addition to the debug profiles available on the Apple Developer website, AppleSeed also includes debug profiles for specific technologies on which they would like to focus testing.

AppleSeed for IT requires that your organization is enrolled in Apple Business Manager or Apple School Manager. Anyone in your organization who has a Managed Apple Account (formerly known as a Managed Apple ID) can participate (except for those with a Student role under Apple School Manager).

To learn more about AppleSeed for IT and to find out how to access these services and tools, go to the AppleSeed site and sign in with your Managed Apple Account.

According to the terms and conditions, you should not share software you download from AppleSeed with anyone else—even in your organization. The best way to have others participate in testing is to have them sign in and download the software using their own Managed Apple Accounts. (More on that in a moment.)

Installing Beta Software

To install betas for macOS Sequoia or iOS/iPadOS 18, you’ll need an Apple account that’s either subscribed to the public beta program or is registered with the Apple Developer Program. It doesn’t have to be a Managed Apple Account, and it may not be the same as the account you use to sign in to iCloud or other Apple services.

Once you have the appropriate ID, your path depends on the OS you want to install. For iOS/iPadOS, you’d go to Settings > General > Software Update > Beta Updates and select AppleSeed Beta. On macOS, you open System Settings, navigate to General > Software Update, click the Info button next to Beta Updates, then choose the beta you want.

In either case, choose the Developer Beta for the upcoming operating system for the device. This will allow you to install the latest beta over the air. Subsequent betas will be available on the device when Apple releases them.

If you want to want to test upgrading or other scenarios that benefit from a more “natural” environment, you can enroll a device running the current operating system. To test on a completely clean system (i.e. one with nothing but the current OS on it), you can download IPSWs for all platforms except watchOS from AppleSeed for IT, then use Apple Configurator to install those system images on target devices. 

Whichever operating system you’re testing, don't install it on production Apple devices. 

Planning your Testing

What’s the best way to test a prerelease OS? The answer obviously depends on your specific circumstances: the kinds and quantity of Apple hardware you manage, the apps and services your organization uses, how much time you have, and so on. But here are some general guidelines to think through.

1) Use Multiple Platforms

Before you do any testing, you need to settle on a test platform—or, more accurately, platforms. Our recommendation: Do it in stages.

Start by installing the new OS on a clean, wiped Mac, iPhone, or iPad. These systems should be dedicated to testing, not in production use. As much as possible, configure the test platform as you would any new production computer or device, with your standard Wi-Fi and other settings, then run it through your suite of tests.

That done, try upgrading a test system that’s currently running the last-generation OS: from macOS Sonoma to macOS Sequoia on the Mac, from iOS or iPadOS 17 to 18 on iPhone or iPad. Once again, configure these devices as you would a production system, then run them through your test suite.

In the past, many admins used virtual machines to test beta OSes. But we don’t recommend that, because VMs have their limits and don’t always produce reliable results. We recommend using actual hardware instead.

2) Start with the OS Itself

You want to take a tiered approach for each platform, starting with the new operating system itself. Does it install? Does it work? Can you connect to local networks (wired and wireless) and the internet? Does Bluetooth work?

If the new OS doesn’t work on a clean Mac, there might be a known issue on the Apple side. Check the release notes on AppleSeed for IT to see if anything there might explain the problem, then provide feedback (see below).

3) Test Your Apps

Once you are comfortable with the operating system's fundamentals, it’s time to test applications. What are your organization’s five most critical user-facing tools? Focus on productivity apps—the Adobe Creative suite, for example—on which your organization relies. Avoid email and calendaring for now, as they depend on back-end infrastructure, which you’ll test next. 

Install these apps on your test hardware and run them through their paces. Does the app install and launch under the new OS? Does it work as you expect? If either of those answers is no, contact the developer of the app in question and let them know. File a report with Apple, as well.

4) Check Your Infrastructure

Next, test the more complex apps and services that your organization relies on. 

Can the test system successfully connect to your email services? If you use Exchange, for example, does it function as expected? What about other cloud services, such as your VPN and your identity provider? Do your security tools work? Security apps, in particular, frequently break with new operating systems.

One special component of that infrastructure that you should test: Your MDM solution. Does the test device with the beta OS enroll correctly? Try deploying a current configuration. Does it all work?

Generally, whatever worked with the previous operating system should work with the upcoming one unless a particular feature has been intentionally removed. If that isn’t the case, you should report the unexpected behavior to Apple and your MDM vendor. Most such vendors don’t make functionality that is new to the upcoming OSes available until Apple officially ships those operating systems. That can make it difficult to test new features. But remember that, with most MDMs, you can craft a custom profile and have the MDM distribute it for you.

Test Plans from Apple

One great resource for test plans is Apple itself. The company supplies sample plans as part of its AppleSeed for IT program. On the AppleSeed site, click Resources, then navigate to the Test Plans & Documentation section. 

The macOS Testing Template there is a zipped Excel spreadsheet containing outlines for a wide variety of testing scenarios, from the deployment and management of Apple devices to office infrastructure. The sheet includes test cases that fill in some of the details for you; they’re incredibly thorough. New plans often appear later in the beta cycle, so it is worth checking this site periodically.

You will likely find items in Apple’s plans you’ve never even considered testing. Even if you don’t need to test the specific scenarios or conditions covered by Apple’s test plans, they’re excellent templates to use in planning tests of your own.

How Much Testing Should You Do?

If your time or resources are limited, keep your plan simple: Test the new OS to see whether it installs correctly on your hardware. If you still have a mix of Intel-based and Apple silicon Mac computers, test the new OS on both. See if you can create a new user and enroll in your MDM solution. If you can take care of those basics, that might be enough for now.

We can’t stress enough how important it is to do as much testing as you can before you roll out a new OS to your users. But if you absolutely don’t have the time or resources for even the most minimal testing program, the fallback is to wait for the OS to be released to the public. When it is, install it on one Mac, and see what (if anything) breaks. In the meantime, keep an eye on public forums (such as the Mac Admins Slack channel) to see what your peers are saying. 

Ideally, you wouldn’t do the testing by yourself. If possible, it’s wise to assemble a testing group—including trusted end-users as well as members of your IT team—to download and test the software themselves. Your MDM solution may provide some tools to manage that process; here’s how Kandji can help.

At the same time, it’s wise to send a message to your organization as a whole warning anyone outside your chosen testing group against downloading pre-release software independently. Your device management solution may be able to help with that, too. Here's how to restrict access to beta OS releases in Kandji.

Providing Feedback

While your main goal of prerelease testing is to check the forthcoming operating systems against your existing hardware and software, there’s another important one: Providing feedback to Apple. For that, your best bet is AppleSeed for IT. 

When to submit feedback? Whenever an app or service you’ve used in the past doesn’t work or behaves oddly—crashing, hanging, or being so slow as to be unusable—under the new OS.

Feedback provided to Apple and app vendors earlier in the beta cycle is more valuable than feedback provided later. It gives Apple and ISVs more time to diagnose and address problems, making it more likely those problems won’t be present in the shipping operating system.

You submit that feedback to Apple through the Feedback Assistant app that’s built into the prerelease OSes. Sign in to Feedback Assistant using the same Managed Apple Account you use to sign in to AppleSeed. File feedback for your organization (rather than Personal) to ensure it gets routed to the AppleSeed for IT triage team. You should be prepared to submit device logs along with any feedback you provide. Along with your description of the problem and the time you encountered it, this gives Apple’s software engineers a starting point for diagnosing the problem.

When you file feedback, make it useful. The more information you can provide, the more likely your bug will get fixed fast and won’t require a lot of follow-up. As a start, include a detailed, descriptive title when filing the report, following the rubrics “[[service or app]] [[is doing this weird thing]]” or “[[can’t do this]] in [[service or app]].” Screenshots are also really helpful. For more, see Apple’s article on how to file great bug reports.

If the problem you’ve identified involves third-party software, make sure to use that vendor’s feedback mechanism as well. It can be difficult at this stage to tell whether a problem is with the operating system or the app, so you’re better off filing a report with both Apple and the ISV.

As mentioned above, if you file feedback using your organization credentials, it will be routed to a special AppleSeed for IT queue. That means your bug submissions and enhancement requests will go to people at Apple who can actually do something about them. 

If you do submit an issue, note the Feedback ID number you receive; it will be used in any follow-up between you and Apple. You can review the feedback you’ve filed or drafted in Feedback Assistant and on the AppleSeed website. Apple will often ask for additional information. These conversations will happen in Feedback Assistant. Your prompt response to their questions makes it more likely that issues can be addressed before release.

One thing to keep in mind when considering whether to file feedback is that when you do so, you’re helping yourself and the community of Apple admins and users. Multiple reports of a problem may help Apple prioritize one issue over another. You may find an issue that another admin didn’t but which will affect them later. Apple has these pre-release periods both to give customers time to react and adapt to changes and to ensure software quality. Your feedback is critical to the company’s efforts. 

One final note: Just because this is the beta season for Apple’s next generation of operating systems doesn’t mean this is the only time you should be testing upcoming releases. Apple is releasing updates to its OSes year-round. You should make testing them before deploying them part of your regular routine. 

About Kandji

By testing betas and leveraging an MDM solution, you can ensure your organization’s transition to the new operating systems is smooth. Kandji supports deploying beta profiles, so you can remotely install those prerelease OSes on devices in a testing group. With powerful features like zero-touch deployment, one-click compliance, and offline remediation, Kandji has everything you need to enroll, configure, and secure your devices.