Debugging Segmentation Faults

This document provides a practical walkthrough of debugging segmentation faults in C programs using core files and GDB debugger, demonstrating commands like backtrace, up, list, and print to analyze crashes, examine variables, and identify common errors like off-by-one array access.

Introduction

Segmentation faults represent one of the most common crash types in C and C++ programs. Understanding how to debug these crashes using core files and the GDB debugger is essential for diagnosing memory access violations and fixing the underlying code issues.

Core Files

What Are Core Files

Core files (also called core dumps) capture a snapshot of a program’s memory state at the moment of a crash. They contain:

• Memory contents of the crashed process
• Register values at crash time
• Stack traces showing function calls
• Variable values when the crash occurred

Purpose of Core Files

Enabling Core File Generation

Using ulimit Command

By default, many systems don’t generate core files or limit their size. Enable unlimited core file generation:

1# Enable core file generation with unlimited size
2ulimit -c unlimited
3
4# Verify the setting
5ulimit -c

Making ulimit Persistent

To make this setting permanent:

1# Add to ~/.bashrc or ~/.bash_profile
2echo "ulimit -c unlimited" >> ~/.bashrc
3
4# Or edit /etc/security/limits.conf (system-wide)
5# Add the following line:
6# * soft core unlimited

Core File Location

Core files are typically generated in:

• Current working directory (default)
• /var/crash/ (some Linux distributions)
• Location specified in /proc/sys/kernel/core_pattern

1# Check core pattern configuration
2cat /proc/sys/kernel/core_pattern
3
4# List core files in current directory
5ls -lh core*

Debugging with GDB

Starting GDB with Core File

Launch GDB with both the executable and core file:

1# Syntax: gdb -c <core-file> <executable>
2gdb -c core example
3
4# Alternative syntax
5gdb example core

Initial GDB Output

When GDB loads a core file, it displays:

• GDB version and license information
• Signal that caused the crash (e.g., SIGSEGV for segmentation fault)
• Location where crash occurred
• Warning messages about missing debugging symbols

GDB Commands for Crash Analysis

Essential GDB Commands

Backtrace Analysis

The backtrace command reveals the call stack at crash time:

1(gdb) backtrace
2#0  strlen () at strlen.c:45
3#1  0x00005555555551a9 in copy_parameters (argc=1, argv=0x7fffffffe0c8) at example.c:10
4#2  0x00005555555551e8 in main (argc=1, argv=0x7fffffffe0c8) at example.c:16

Reading backtrace frames:

• Frame #0: Where crash occurred (deepest function)
• Frame #1: Function that called frame #0
• Frame #2: Function that called frame #1 (often main)

Navigating Stack Frames

 1# Move to calling function
 2(gdb) up
 3#1  0x00005555555551a9 in copy_parameters (argc=1, argv=0x7fffffffe0c8) at example.c:10
 410          len = strlen(argv[i]);
 5
 6# View source code context
 7(gdb) list
 85   void copy_parameters(int argc, char *argv[]) {
 96       int i;
107       size_t len;
118
129       for (i = 0; i <= argc; i++) {
1310          len = strlen(argv[i]);
1411          // ... rest of code
1512      }
1613  }

Examining Variables and Memory

Printing Variable Values

 1# Print loop counter
 2(gdb) print i
 3$1 = 1
 4
 5# Print array element
 6(gdb) print argv[0]
 7$2 = 0x7fffffffe3a0 "./example"
 8
 9# Print next array element
10(gdb) print argv[1]
11$3 = 0x0

Understanding Pointer Values

Memory Addresses in GDB

Hexadecimal numbers (starting with 0x) represent memory addresses:

• Valid pointers point to accessible memory
• Null pointer (0x0) is never valid
• Accessing null pointers causes segmentation faults

Case Study: Off-by-One Error

Problem Analysis

The example crash occurs in a for loop iterating over command-line arguments:

1for (i = 0; i <= argc; i++) {
2    len = strlen(argv[i]);
3    // Process argument
4}

Identifying the Bug

GDB investigation reveals:

1(gdb) print i
2$1 = 1
3
4(gdb) print argv[0]
5$2 = 0x7fffffffe3a0 "./example"
6
7(gdb) print argv[1]
8$3 = 0x0  # NULL pointer!

The loop condition i <= argc causes one iteration too many:

• argc = 1 (one argument: program name)
• Array has elements at indices 0 (valid) and 1 (NULL terminator)
• Loop iterates with i=0 (valid) and i=1 (NULL pointer access)

The Fix

Change the loop condition from less-than-or-equal to strictly less-than:

1// Before (buggy)
2for (i = 0; i <= argc; i++) {
3    len = strlen(argv[i]);
4}
5
6// After (fixed)
7for (i = 0; i < argc; i++) {
8    len = strlen(argv[i]);
9}

Off-by-One Errors

Important

Off-by-one errors are among the most common bugs in C programming. Always carefully verify loop boundaries and array indices.

Debugging Symbols

Missing Debug Symbols

GDB may display warnings about missing debugging symbols:

1Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...
2(No debugging symbols found in /lib/x86_64-linux-gnu/libc.so.6)

Installing Debug Symbols

1# Debian/Ubuntu
2sudo apt-get install libc6-dbg
3
4# Red Hat/Fedora
5sudo yum install glibc-debuginfo
6
7# Install debug symbols for specific package
8sudo apt-get install <package>-dbg

When Debug Symbols Matter

Complete Debugging Workflow

Step-by-Step Process

 1# 1. Enable core dumps
 2ulimit -c unlimited
 3
 4# 2. Reproduce the crash
 5./example
 6# Segmentation fault (core dumped)
 7
 8# 3. Verify core file created
 9ls -lh core
10
11# 4. Start GDB with core file
12gdb -c core example
13
14# 5. Get backtrace
15(gdb) backtrace
16
17# 6. Navigate to relevant frame
18(gdb) up
19
20# 7. View source code
21(gdb) list
22
23# 8. Examine variables
24(gdb) print i
25(gdb) print argv[i]
26
27# 9. Analyze the problem
28# Identify root cause
29
30# 10. Exit GDB
31(gdb) quit
32
33# 11. Fix the code
34# Edit source file
35
36# 12. Recompile with debug symbols
37gcc -g -o example example.c
38
39# 13. Test the fix
40./example

Best Practices

Compile with Debug Symbols

Always use -g flag during development:

1# Add debug symbols
2gcc -g -o program program.c
3
4# Add debug symbols with optimization (harder to debug)
5gcc -g -O2 -o program program.c
6
7# Debug symbols without optimization (recommended for debugging)
8gcc -g -O0 -o program program.c

Core File Management

• Set appropriate size limits with ulimit -c
• Monitor disk space (core files can be large)
• Configure core file naming patterns
• Archive important core files for later analysis

Debugging Tips

• Start with backtrace to understand call chain
• Move up frames to find your code
• Examine variables and their values
• Check array bounds and pointer validity
• Look for null pointers (0x0)
• Verify loop conditions and boundaries

Note

Trust standard library functions like strlen(). If they crash, the problem is almost always in the calling code (invalid arguments).

Conclusion

Debugging segmentation faults requires generating core files with ulimit -c unlimited, analyzing them with GDB using commands like backtrace, up, list, and print, and carefully examining variable values and memory addresses. Off-by-one errors in loops and array access are common causes of segmentation faults. Systematic analysis of the call stack and variable states leads to identifying and fixing the root cause.

FAQ