The relationship between processor and operating system is crucial. If an OS can’t properly manage the tasks set before it, it doesn’t matter how fast the CPU runs.
At a private press and analyst event known as Architecture Day, Intel unveiled the hardware-level feature that would give it an edge over its rivals and fully utilize the advantages of Alder Lake’s hybrid architecture. It’s called Thread Director, a way of creating an unprecedented connection between the processor and the operating system — or in this case, Alder Lake and Windows 11.
Thread Director isn’t the first of its kind, and neither is the Alder Lake hybrid computing model. However, by adding Windows 11 into the mix, Intel is promising improved performance and efficiency across nearly all workloads. This is new for much of the desktop and laptop market, and it’s going to be a big focus for Intel moving forward, so let’s get up to speed.
The need for scheduling
When you use your computer, you never have to think about the thousands of various tasks it’s handling. That’s because the OS does it for you. Even just writing this post with some extra tabs open, my PC is running about 3,500 threads at the same time, none of which I need to monitor or think about.
The OS does it for me through a scheduler. A scheduler assigns resources to tasks (such as a thread to a core), and up to this point, it has relied on certain static information — such as if the task is in the foreground or background — and guesswork to assign tasks to the right core. Intel’s new Thread Director technology changes that.
Rajshree Chabukswar is an Intel client architect that helped design Thread Director, and I had a chance to talk with her a bit more about how the technology works and why it’s necessary. This “special and unique” technology, as Chabukswar put it, is the key to unlocking performance between a processor and an OS, and it hinges on a hybrid CPU architecture.
“Think about what’s running on our laptops,” Chabukswar said. “There is a bunch of background activity that happens, different tasks with different performance expectations. It doesn’t make sense to have a cookie-cutter approach.”
A cookie-cutter approach would be the same cores running tasks in whatever way the OS decides, but Alder Lake is different. Its hybrid architecture combines high-performance and high-efficiency cores together, similar to a lot of ARM processors in mobile devices. And Thread Director gives the OS visibility into which of those cores are better for a given task.
“Our Thread Director hardware looks at various combinations of various performance monitoring units, then provides one number or hint to the operating system.” The OS then takes that hint and applies its own decision on top, ultimately deciding if a task should go to a performance core or an efficiency one.
“The goal was clear: How do we get the best performance out of Alder Lake?”
Although Alder Lake isn’t the first hybrid CPU, and Thread Director isn’t the first hardware scheduling tool, the combination of them is something unique, according to Intel. Chabukswar summed it up nicely: “Thread Director can detect [inefficiencies] and say ‘hey, I see that this is not doing useful work, so I’m not going prioritize it for the most performant core because I have some other work lined up for that.”
Thread Director is only one piece of the puzzle, though. To fully utilize hardware scheduling, it needs to integrate with the OS’s built-in scheduler. And for Windows 11, that’s exactly what Microsoft and Intel are doing.
A unified front with Windows 11
About two years ago, Intel and Microsoft decided to expand on their “many, many, many” years of close partnership to have Alder Lake and Windows 11 operating as a single unit. “The goal was clear: How do we get the best performance out of Alder Lake?”
It’s not hard to imagine that Microsoft has a similar goal for Windows 11. Though they work with other CPU providers such as AMD and Qualcomm, Intel is undoubtedly the most important partner Microsoft has. If Intel was all-in on the hybrid computing model, encouraging and supporting features like Thread Director would be a no-brainer.
Principal software engineer at Microsoft, Chris Kleynhans, described the relationship this way:
“Early on in Windows 11 development, Intel approached us with a proposal for an interface that would allow the CPU to guide the operating system scheduler by providing information about how much a specific workload would benefit from being scheduled to a performance core instead of an efficiency core.”
According to him, the proposal kicked off a deep collaboration during the early phases of Windows 11’s development, incorporating Thread Director feedback right into the thread scheduler. Even though Microsoft says its changes to Windows 11 will help hybrid CPUs in general (including processors from Intel’s rivals), it’s clear that Intel’s implementation is something special. The stakes were high for Intel, and it needed a hardware solution baked-in to Alder Lake if they were going to take full advantage of what hybrid computing can do.
Thankfully, processors already have information about what kind of instructions each task calls for, and by offering Windows 11 visibility into that, Intel is able to provide the built-in scheduler information about which instructions are most important.
“Hardware naturally has that information available,” Intel’s Chabukswar explained. “It’s in our performance monitoring unit, so it made sense to encapsulate that in hardware and provide that hint to the operating system.”
According to Chabukswar, the pieces fell as they should. Hybrid made sense for the increasing demands of power and performance, and hardware-level scheduling hints made sense for an OS to take full advantage of each of the cores. Recent ARM processors already do something similar across various platforms, so it makes sense for traditional desktop and laptop scenarios, too.
It was a “natural transition” as Intel started talking about Alder Lake and its hybrid architecture. After presenting Alder Lake, it seems that Microsoft had no qualms about Thread Director and how it could benefit the upcoming Windows 11. “Microsoft understood the value proposition and was immediately brought into that.”
Windows 11 is the focus of Thread Director, and during my talk with Chabukswar, it was clear that the OS and Thread Director need to work together for full optimization. However, Windows 10 should benefit from Thread Director as well, and down the line, other operating systems will see a benefit, too.
“In future revisions, we will add more capabilities,” Chabukswar told me.
Optimizing power and performance
ARM pioneered the big.LITTLE hybrid architecture design and ARM processors also take advantage of the OS scheduler to improve performance. However, it sounds like the current implementation works a little differently than Thread Director. In ARM’s case, the OS knows the computational power of each core and assigns work to them based on multiple tunable thresholds.
Thread Director makes decisions based on the instructions the current task requires in the context of all tasks running. A task with an A.I. instruction set, for example, would need a high-performance core. Given the same set of instructions across tasks, Thread Director does effectively nothing, as there’s no difference in the tasks being executed.
“If you look at the typical hybrid implementation that’s in the ecosystem right now, they make claims about what will be best on power at the cost of performance. We didn’t want to sacrifice.”
The goal with Thread Director was to create a dynamic scheduling feature that could adapt to workloads on the nanosecond level. “Our approach here was ‘how do we make it non-static?’ It’s truly dynamic in my mind.”
In the past, dynamic scheduling optimization has mostly focused on efficiency, which makes sense. ARM, which has pushed this idea, is mainly designing chips for devices that call for long battery life and high efficiency. Alder Lake, on the other hand, is a scalable architecture that Intel intends to use from high-performance desktops to highly efficient embedded mobile solutions.
“The goal with Alder Lake was not just energy efficiency. If you look at the typical hybrid implementation that’s in the ecosystem right now, they make claims about what will be best on power at the cost of performance. We didn’t want to sacrifice.”
Although Thread Director helps improve performance, it can also help increase efficiency. That comes down to Thread Director, as well as a quality-of-service (QoS) application programming interface (API) that Microsoft has had for a few years. This API allows developers to set throttling policies on processes, essentially allowing unimportant tasks to consume less power.
With the API, Thread Director is able to give better instructions to the OS. Chabukswar provided two examples where this is useful. You could have multiple tabs open in a web browser, for example, and one of those tabs has an animation playing that’s not important. It’s not on the screen, so it doesn’t need priority when scheduling tasks.
Similarly, a background animation in a game, maybe one that’s static and doesn’t impact performance, isn’t a high-priority task. Developers can already tune these tasks to consume less power, and now, they can do so across a hybrid architecture. “Developers can now tell the operating system ‘I know this thread is doing this, but don’t prioritize it to any performant threads.'”
Microsoft Edge in Windows 11 already has this API working, and Microsoft says that it’s in the process of optimizing even more software. “We can enhance the decisions provided by Thread Director using this QoS API,” Chabukswar said.
A paradigm shift?
Alder Lake is exciting for Intel — and it needs to be, with the last few generations tracking downward compared to the competition. With it, Intel promised a breakthrough x86 architecture, and by leveraging hardware-guided scheduling and a hybrid CPU design, it very well could be.
Different doesn’t always mean better, however. It’s true that Alder Lake is a major shift for Intel, especially compared to the recent generations that are derivative of ones that came before. As for if that shift will make a difference when Alder Lake finally launches, we’ll just have to wait and see.
