Using Threads to Improve Performance

This document provides a hands-on guide to implementing threading and multiprocessing in Python for performance optimization. Through a real-world image thumbnail generation scenario, it demonstrates converting sequential processing to parallel execution using ThreadPoolExecutor and ProcessPoolExecutor, measuring performance improvements, and understanding the differences between threads and processes in Python's execution model.

The Business Problem: E-Commerce Image Rebranding

The Scenario

A company has an e-commerce website that includes numerous images of products that are available for sale. An upcoming rebranding effort requires that all of these images be replaced with new ones.

Scope of work:

Existing Infrastructure

There is an existing script that creates thumbnails based on the full-size images. But there are a lot of files to process, and the script is taking a long time to finish.

Current situation:

Baseline Performance Measurement

Test Setup

The process begins by trying out the current script as-is using a set of 1,000 test images. There are more images to convert, but it will be easier to test the speed of the script with a smaller batch.

Test configuration:

Measuring with the Time Command

The program is executed using the time command to see how long it takes.

1time python3 thumbnail_generator.py

Baseline results:

Initial Assessment

It took about two seconds for 1,000 images. This doesn’t seem too slow, but there are tens of thousands of images that need converting, and the goal is to ensure that the process is as fast as possible.

Scaling projection:

The need for optimization is clear when considering the full scale of the task.

Implementing Thread-Based Parallelism

Understanding Python Threading

To make processing go faster by having it process the images in parallel, the implementation starts with importing the futures submodule, which is part of the concurrent module.

Python concurrent.futures module:

This gives a very simple way of using Python threads.

Creating an Executor

To be able to run things in parallel, an executor must be created. This is the process that’s in charge of distributing the work among the different workers.

Executor role:

The futures module provides a couple of different executors: one for using threads and another for using processes.

Initial Implementation: ThreadPoolExecutor

The implementation will start with the ThreadPoolExecutor.

Code structure:

1from concurrent import futures
2
3# Create executor
4executor = futures.ThreadPoolExecutor()

ThreadPoolExecutor characteristics:

Modifying the Processing Loop

Original Sequential Code

Now the function that does most of the work in the loop is process_file.

Sequential pattern:

1for image_file in image_files:
2    process_file(image_file)  # Called directly - sequential

Sequential execution flow:

Parallel Implementation with Executor

Instead of calling the function directly in the loop, a new task is submitted to the executor with the name of the function and its parameters.

Parallel pattern:

1from concurrent import futures
2
3executor = futures.ThreadPoolExecutor()
4
5for image_file in image_files:
6    executor.submit(process_file, image_file)

Parallel execution flow:

Understanding Task Scheduling

The for loop now creates a bunch of tasks that are all scheduled in the executor. The executor will run them in parallel using threads.

Task lifecycle:

Handling Executor Completion

The Asynchronous Nature of Executors

An interesting thing that happens when using threads is that the loop will finish as soon as all tasks are scheduled. But it will still take a while until the tasks complete.

Timing comparison:

Adding Proper Shutdown

A message is added indicating that the script is waiting for all threads to finish, and then the shutdown function is called on the executor.

1from concurrent import futures
2
3executor = futures.ThreadPoolExecutor()
4
5for image_file in image_files:
6    executor.submit(process_file, image_file)
7
8print("Waiting for all threads to finish...")
9executor.shutdown()

Shutdown function behavior:

This function waits until all the workers in the pool are done, and only then shuts down the executor.

Thread-Based Performance Results

Running the Optimized Script

After making the change, the script is saved and tested.

1time python3 thumbnail_generator_threaded.py

Threading results:

The script now takes 1.2 seconds. That’s a nice improvement over the two seconds observed before.

Understanding User Time vs Real Time

Notice how the user time is higher than the real time when using threads.

Time metric interpretation:

By using multiple threads, the script is making use of the different processors available in the computer. The user time value shows the time used on all processors combined.

Note

When user time exceeds real time, it indicates successful parallel execution across multiple CPU cores. The user time represents the sum of CPU time used by all threads, while real time is the actual elapsed wall-clock time.

Implementing Process-Based Parallelism

Switching to ProcessPoolExecutor

What will happen if processes are used instead of threads? This can be tested by changing the executor being used.

Simple code change:

1# Before: threads
2executor = futures.ThreadPoolExecutor()
3
4# After: processes
5executor = futures.ProcessPoolExecutor()

By changing the executor to the ProcessPoolExecutor, the futures module is instructed to use processes instead of threads for the parallel operations.

Executor comparison:

Process-Based Performance Results

Running with Processes

After saving the change, the script is tested again.

1time python3 thumbnail_generator_multiprocess.py

Process-based results:

This is now taking less than a second to finish, and the user time has gone up even more.

Why Processes Are Faster

This is because, by using processes, even more use is being made of the CPU.

CPU utilization comparison:

Understanding the Thread vs Process Difference

Python’s Global Interpreter Lock (GIL)

The difference is caused by the way threads and processes work in Python.

The GIL mechanism:

Thread Synchronization Overhead

Threads use a bunch of safety features to avoid having two threads that try to write to the same variable.

Thread safety mechanisms:

And this means that when using threads, they may end up waiting for their turn to write to variables for a few milliseconds, adding up to the small difference between the two approaches.

Process Independence

Process advantages:

When each approach excels:

Important

For CPU-bound tasks like image processing, ProcessPoolExecutor provides true parallel execution and significantly better performance than ThreadPoolExecutor. For I/O-bound tasks, ThreadPoolExecutor is usually sufficient and has lower overhead.

Performance Comparison Summary

All Approaches Compared

Complete performance analysis:

Scaling to Full Workload

Projected performance for full image set:

The multiprocess approach saves significant time at scale.

Additional Optimization Opportunities

Further Improvements

There are still more improvements that can be made to the script.

Potential enhancements:

Checking Thumbnail Status

Adding a check to see if the thumbnail exists and is up to date before doing the conversion could significantly reduce work.

Smart processing logic:

1def should_process(image_file, thumbnail_file):
2    if not os.path.exists(thumbnail_file):
3        return True  # Thumbnail doesn't exist
4
5    image_mtime = os.path.getmtime(image_file)
6    thumb_mtime = os.path.getmtime(thumbnail_file)
7
8    return image_mtime > thumb_mtime  # Source newer than thumbnail

Impact of smart checking:

Progress Feedback

Adding a progress bar while waiting for tasks to finish makes it clear that the script is doing its job.

Progress bar benefits:

Implementation Best Practices

Executor Pattern Guidelines

Code Example with Best Practices

 1from concurrent import futures
 2import os
 3
 4def process_file(image_file):
 5    """Process a single image file to create thumbnail"""
 6    try:
 7        # Create thumbnail logic here
 8        thumbnail_file = get_thumbnail_path(image_file)
 9
10        # Skip if already up to date
11        if is_thumbnail_current(image_file, thumbnail_file):
12            return f"Skipped: {image_file}"
13
14        # Generate thumbnail
15        create_thumbnail(image_file, thumbnail_file)
16        return f"Processed: {image_file}"
17    except Exception as e:
18        return f"Error processing {image_file}: {e}"
19
20# Use context manager for automatic cleanup
21with futures.ProcessPoolExecutor(max_workers=os.cpu_count()) as executor:
22    # Submit all tasks
23    future_to_file = {
24        executor.submit(process_file, img): img
25        for img in image_files
26    }
27
28    # Process results as they complete
29    for future in futures.as_completed(future_to_file):
30        result = future.result()
31        print(result)

Key Takeaways

Threading vs Multiprocessing Decision Matrix

Performance Optimization Workflow

Conclusion

This hands-on example demonstrated practical implementation of threading and multiprocessing in Python for performance optimization of an image thumbnail generation script. The baseline sequential processing took 2 seconds for 1,000 images, which would scale poorly to the full workload of tens of thousands of images. Implementing ThreadPoolExecutor by importing the concurrent.futures module, creating an executor, and submitting tasks instead of calling functions directly improved performance to 1.2 seconds (40% faster), with the executor.shutdown() call ensuring all tasks complete before script exit. Switching to ProcessPoolExecutor with a single line change (replacing ThreadPoolExecutor with ProcessPoolExecutor) further improved performance to under 1 second (2× faster than baseline), demonstrating the superiority of processes for CPU-bound tasks. The performance difference between threads and processes stems from Python’s Global Interpreter Lock (GIL), which causes threads to wait for their turn to execute bytecode, introducing millisecond delays that accumulate across many operations, while processes each have their own GIL and achieve true parallelism. The user time exceeding real time when using parallelism indicates successful multi-core utilization, with the user time representing combined CPU time across all cores. For CPU-bound tasks like image processing, ProcessPoolExecutor is the superior choice, while ThreadPoolExecutor works well for I/O-bound operations where the GIL is released during I/O waits. Additional optimization opportunities include checking if thumbnails already exist and are current before processing, adding progress bars for user feedback, implementing proper error handling per image, and using context managers for automatic executor cleanup. The demonstrated executor pattern—submit tasks in a loop, then shutdown to wait for completion—provides a simple yet powerful approach to parallelism that can dramatically improve performance for batch processing tasks with minimal code changes.

FAQ