How and Why You Should Be Testing Apple's Next Operating Systems Now

We said it last year, but we’ll say it again: Now that prerelease versions of Apple’s next operating systems—this year, that means macOS Ventura and iOS and iPadOS 16—are available to the public, it’s time for Apple admins to be testing them. 

The reason is simple: You need to evaluate a new OS before deploying it widely, so you know how well (or if) it will get along with your existing technical infrastructure. 

But it’s one thing to know you need to test and another to get that testing done. Here are a few guidelines for ensuring that you'll be ready when those OSes finally do ship this fall.

Downloading Prerelease Operating Systems

The first step in testing is to actually acquire the software. Apple distributes beta versions of its operating systems through several channels, including its Beta Software Program, Apple Developer, and AppleSeed for IT programs. All three let users, developers, and IT professionals test the compatibility of new OS versions with their apps and environments; in return, Apple asks only that those users submit feedback.

But only one of the three is specifically designed for IT admins who need to test new versions of Apple beta software in managed environments: AppleSeed for IT. Participating in AppleSeed is the best way to set up your organization for success when the latest OSes are released.

AppleSeed for IT provides admins with:

• Access to prerelease software: AppleSeed lets you test operating systems and select Apple apps in your business environment before they’re released.
• Detailed test plans and surveys: Apple supplies test plans—including regression and new-feature tests—to help you evaluate the compatibility of beta software with your existing tech infrastructure. 
• Dedicated review queue: When you submit feedback through AppleSeed, it’s funneled to a dedicated review queue to make sure bug submissions and enhancement requests get to the right people at Apple as soon as possible.
• Discussion forums: When you join AppleSeed, you gain access to its exclusive discussion forums. They’re a great place to post questions, help other participants, and discuss other IT topics. 

AppleSeed for IT requires that your organization is enrolled in Apple Business Manager or Apple School Manager. Anyone with a Managed Apple ID in your organization can participate (except for those with a Student role under Apple School Manager).

To learn more about AppleSeed for IT and to find out how to access these services and tools, go to appleseed.apple.com and sign in with your Managed Apple ID.

According to the terms and conditions, you should not share software you download from AppleSeed with anyone else—even in your organization. The best way to have others participate in testing is to have them sign in with their own Managed Apple IDs.

Installing Beta Software

To install the beta software, you’ll need to install the macOS, iOS, or iPadOS beta software profile on your device. 

macOS: Download the macOS Customer Beta Access Utility from the Downloads tab in the AppleSeed portal. After you install it, the Feedback Assistant app and System Preferences will launch automatically. You can then download and install the macOS beta from the Software Update pane in System Preferences.

iOS and iPadOS: If there was a previous beta software profile on your device, remove it (Settings > General, then Device Management or Profiles depending on the configuration). You then download the Beta Software Profile from the Downloads tab of the AppleSeed portal. When that download is complete, you should be prompted to install it. After a restart, it should be in place.

Once your test device is running the iOS beta, new versions of it can be installed over the air; you should get notifications about new builds via email and on the device. You should then be able to open Settings > General > Software Update to download and install the latest version.

Whichever operating system you’re testing, don't install it on production Apple devices. Better to create a beta group in your MDM solution to configure your test devices with apps and settings to test against the beta OS. (In Kandji, you can do this by uploading the AppleSeed for IT beta profile as a Custom Profile, then assigning it to a Blueprint that goes out to your testing group.)

Note: If Apple offers an IPSW file to use with Apple Configurator to restore an Apple device to the latest beta with Apple silicon, you might find that convenient.

Planning your Testing

What’s the best way to test a prerelease OS? The specific answers obviously depend entirely on your specific circumstances: the kinds and quantity of Apple hardware you manage, the apps and services your organization uses, how much time you have, and so on. But here are some general guidelines to think through.

1) Use Multiple Platforms

Before you do any testing, you need to settle on a test platform—or, more accurately, platforms. Our recommendation: Do it in stages.

Start by installing the new OS on a clean, wiped Mac, iPhone, or iPad. These systems should be dedicated to testing, not actually in production use. As much as possible, configure the test platform as you would any new production computer or device, with your standard Wi-Fi and other settings, then run it through your suite of tests.

That done, try upgrading a test system that’s currently running the last generation OS: from macOS Monterey to macOS Ventura on the Mac, from iOS or iPadOS 15 to 16 on iPhone or iPad. Once again, configure these devices as you would a production system, then run them through your test suite.

Many admins use virtual machines for testing. VMs can indeed be a boon but be warned: Because they don’t always replicate physical machines perfectly, VMs can have their limitations as testing platforms. (Note that, with some recent announcements by Apple, some of those limitations are likely to go away.) Virtual machines can be a strong component of a testing plan, but shouldn’t be the only one.

2) Start with the OS Itself

You want to take a tiered approach for each platform, starting with the new operating system itself. Does it install? Does it work? Can you connect to local networks (wired and wireless) and the internet? Does Bluetooth work?

If the new OS doesn’t work on a clean Mac, there might be a known issue on the Apple side. Check the release notes on AppleSeed for IT to see if there are known issues that might explain the problem and provide feedback (see below).

3) Test Your Apps

Assuming you can get the beta operating system to install and work, turn next to applications. What are your five most important user-facing tools? Focus on productivity apps—Microsoft Office or the Adobe Creative suite, for example—on which your organization relies. Avoid email and calendaring for now, as they depend on back-end infrastructure that you’ll test next. 

Install these apps on your test hardware and run them through their paces. Does the app install and launch under the new OS? Does it provide the functionality you expect? If any of those answers are no, contact the developer of the app in question and let them know. File a report with Apple, as well.

4) Check Your Infrastructure

Next, test the more complex apps and services that your organization relies on. 

Can the test system connect successfully to your email services? If you use Exchange, for example, does it function as expected? What about other cloud services, such as your VPN and your identity provider? Do your security tools work? (Security apps in particular frequently break with new operating systems.)

One special component of that infrastructure that you should test: Your MDM solution. Does the test device with the beta OS enroll correctly? Try deploying a current configuration. Does it all work? Next, can you use your MDM to deploy the beta OS to the test hardware? Again, try it first on a freshly wiped system, then one that you’re upgrading from a previous OS.

Test Plans from Apple

One great resource for test plans: Apple itself. The company supplies sample plans as part of its AppleSeed for IT program. (In the AppleSeed site, go to the Test Plans and Additional Resources section of the Downloads tab.) These plans cover a wide range of topics, including Microsoft Exchange, managed software updates, data separation, and more. They're also incredibly thorough.

So, for example, Apple provides a sample plan for testing macOS on Apple silicon. It covers the testing of your MDM solution, your network infrastructure, and apps, and it steps you through a variety of test cases.

You will likely find items in Apple’s plans you’ve never even considered testing. Even if you don’t need to test the specific scenarios or conditions covered by Apple’s test plans, they’re excellent templates to use in planning tests of your own.

How Much Testing Should You Do?

If your time or resources are limited, keep your plan simple: Test the new OS to see whether it installs correctly on your hardware. (If you have a mix of Intel-based and Apple silicon Mac computers, test the new OS on both.) See if you can create a new user and enroll in your MDM solution. If you can take care of those basics, that might be enough for now.

We can’t stress enough how important it is to do as much testing as you can before you roll out a new OS to your users. But if you absolutely don’t have the time or resources for even the most minimal testing program, the fallback is to wait for the OS to be released to the public. When it is, install it on one machine, and see what (if anything) breaks. In the meantime, keep an eye on public forums (such as the MacAdmins Slack channel) to see what your peers are saying. 

Providing Feedback

While the main goal of prerelease testing is to check the forthcoming operating systems against your existing hardware and software, there’s another crucial goal: Providing feedback to Apple about the new OSes. For that, your best bet is AppleSeed for IT. 

When to submit feedback? Whenever an app or service you’ve used in the past doesn’t work or behaves oddly—crashing, hanging, or being so slow as to be unusable—under the new OS.

You submit that feedback through the Feedback Assistant app that’s built into the prerelease OSes. You should be prepared to submit device logs along with any feedback you provide. 

When you file feedback, it's important to make it useful. Don't just say, "It doesn't work". The more information you can provide, the more likely your bug will get fixed fast and won’t require a lot of follow-up. At a minimum, include a detailed, descriptive title when filing the report, following the rubrics “[[service or app]] [[is doing this weird thing]]” or “[[can’t do this]] in [[service or app]].” Screenshots are also really helpful. For more, see Apple’s article on how to file great bug reports.

Whichever way you submit feedback, it will be routed to a special AppleSeed for IT queue. That means your bug submissions and enhancement requests will go to people at Apple who can actually do something about them. 

If you do submit an issue, note the Feedback ID number you receive; it will be used in any follow-up between you and Apple. You can review the feedback you’ve filed or drafted in Feedback Assistant and on the AppleSeed website.

One final note: While you should be downloading and checking prerelease software in a dedicated testing environment, you probably don't want your users to do the same thing on their production devices. Your device management solution may be able to help with that. Here's how to restrict access to beta OS releases in Kandji.

About Kandji

By testing betas and leveraging an MDM solution, you can ensure your organization’s transition to the new operating systems is smooth. Kandji supports deploying beta profiles, so you can remotely install those prerelease OSes on devices in a testing group. With powerful features like zero-touch deployment, one-click compliance, and offline remediation, Kandji has everything you need to enroll, configure, and secure your devices.