Sunday, March 1, 2015

CPU Basics: Processes, Threads & Cores

If you've been keeping up with this blog, you've already been introduced to a host of introductory articles covering three disciplines of engineering- programming, system administration, and quality assurance. Now there were times when I dived into deeper topics within these disciplines- monitoring, for example, as a subtopic of system administration. We're at another one of those junctions, but this time we're going to cover benchmarking and profiling. To kick off this series, we're going over how the CPU allocates resources to different applications running on the kernel, to ensure that all of the instructions are carried out as efficiently as possible.

Processes and Threads

Programs, or applications, are just a series of steps that need to be carried out in order to perform a specified task. Each step ends up being converted into an instruction, each with a designated binary code unique to the CPU. So when we talk about processes, we're referring to an instance of an application that was started as a result of the user double clicking the Chrome icon on their desktop, for instance. And a thread is a path of execution within a process. Note, a process can contain multiple threads.

Now, a thread can do anything a process can do but since a process can contain multiple threads, threads are regarded as lightweight processes. Another thing to note is that threads within the same process share the same address space. Different processes do not. Sharing that space allows threads to communicate and read and write to the same data structures and variables. For processes to communicate with each other, expensive resources need to be allotted to IPC, or inter-process communication.


The CPU, or central processing unit, is the set of electronic components (resistors, transistors, capacitors, inductors, and diodes) responsible for carrying out a program's operations. Within this system, you're going to find system memory (RAM) for storing binary codes to be executed, the arithmetic logic unit (ALU), and hardware registers. Most of today's CPUs often reside alongside other CPUs on the same chip to form a multi-core processor.

Preparing a Program for Execution

When an instruction needs to be carried out, the CPU must access RAM through a bus. A bus is just a channel for transferring data between components within a computer or network. Since we're focusing on the CPU, we're referring to the internal bus (a.k.a. internal data bus, memory bus, system bus or Front-Side bus), which connects the CPU, memory and I/O to the motherboard. Depending on where the program lives, the flow can vary. If the program lives on the hard drive, for example, the CPU must first transfer it to system memory. The CPU then reads the program through one of the computer buses (address, data, control) to move these bits into the CPU.

The Pipeline

Now that the program is in the CPU, it's deposited into a queue for processing. Instructions are then distributed across various components for processing. This process is called the Machine Cycle.

Now every CPU has its own little heartbeat. It's just a clock that runs at a certain frequency. 2GHz, for instance. This pipeline comes with its own logic for executing every step, or series of instructions, that a program needs to execute. In the days of MS-DOS and command line OSes, this execution flow was much more straightforward. With today's multi-tasking operating systems where you have multiple applications and threads open, simultaneously, it's important to provide the end user with a sense of parallelism. Just one application, alone, could come with threads for keyboard input, fetching data, rendering, saving, and so on. The computer has a mechanism for handling all of these operations so that you, as a user, feels as if everything is happening concurrently. This is where the scheduler comes in.

The Scheduler

The scheduler is a software component and switch that controls the manner in which instructions are carried out by the CPU. The scheduler will back and forth between different instructions, allocating a period of time, known as time slice, for execution. The CPU can actually execute thousands of instructions per millisecond- so fast that it appears to all be happening simultaneously. This process is also known as preemption or preemptive multitasking for the fact that it's up to the scheduler to dictate control, not the application. The scheduler is constantly interrupting tasks, context switching, and deciding when to resume tasks down the road based on priority and other conditions. This has the added benefit of not allowing an application to crash the OS.


Every thread that will be handled by the scheduler is assigned a priority. It's simply a number that governs the order in which a thread is passed onto the CPU. The scheduler is also in charge of boosting priority of threads that aren't currently being executed. So what results is this constant back and forth of high priority thread execution and low priority thread boosting until all of the tasks are completed.

In future articles, I'll make sure to go over preemptive, non-preemptive, and round robin CPU scheduling in detail.

More Ways to Achieve Concurrency

In conclusion, CPU resources today need to be leveraged in different ways to be able to address demands at scale. In the past, when we were limited to single core CPUs, we could leverage a system known as hyperthreading, where we could duplicate a segment of our pipeline in order to handle multiple threads. Unfortunately, the CPU could only allocate certain components to one thread at a time so threads were often in stasis until those components freed up. Overall, though, this system was slightly faster than one without hyperthreading.

Now, we go so far as to leverage multi-core CPUS and even multi-CPUS. Many of today's services are delivered through high-availability clusters where you need to thinking about load balancing and redundancy. While expensive, this is often your best solution to handle high traffic demands.

That concludes this article. Check the "Related Articles" section, below, if you'd like to learn more about the topic.

No comments:

Post a Comment