Analysis and Solution for Python IOError: [Errno 28] No Space Left on Device

Problem Background and Symptoms

During Python script execution, when attempting to write large numbers of files to external hard drives, developers may encounter the IOError: [Errno 28] No space left on device error. This phenomenon is particularly puzzling because the script may run normally on some machines while failing on others.

A typical scenario involves: the script successfully writes over 30,000 files on the problematic machine before suddenly starting to fail. Disk space checks reveal ample available capacity, ruling out simple disk capacity issues.

Error Code Analysis

The problematic Python code typically involves looped write operations:

nPage = 0
while nPage != -1:
    for d in data:
        if len(d.contents) > 1:
            if '<script' in str(d.contents):
                l = str(d.contents[1])
                start = l.find('http://')
                end = l.find('>', start)
                out = get_records.openURL(l[start:end])
                print COUNT

                with open('../results/'+str(COUNT)+'.html', 'w') as f:
                    f.write(out)
                COUNT += 1

    nPage = nextPage(mOut, False)

This code continuously creates new files within a loop, triggering errors when file counts reach filesystem limitations.

Root Cause Analysis

The ENOSPC error doesn't always indicate physical disk space exhaustion. It can be caused by multiple factors:

• Inode exhaustion: Filesystems allocate an inode for each file to store metadata. When inodes are exhausted, new files cannot be created even with available physical space
• Directory entry limits: Certain filesystems (like vfat) impose limits on the number of files in a single directory
• Filesystem corruption: Long-term usage may lead to filesystem structure corruption
• Temporary file space exhaustion: System temporary directories may be full

Solution: Drive Formatting

Based on practical experience, formatting the external hard drive has proven to be the most effective solution. This approach can:

• Completely eliminate filesystem errors and corruption
• Reset the inode allocation table
• Restore directory structure integrity
• Remove problems accumulated through long-term usage

Always backup important data before formatting, then reformat the drive using an appropriate filesystem. For Linux environments, the ext4 filesystem is recommended as it handles large numbers of small files better than filesystems like vfat.

Alternative Viable Solutions

Besides formatting, the following approaches can be attempted:

• Clean temporary files: Removing temporary files from the /tmp/ directory can free up space
• Check inode usage: Use the df -i command to check inode utilization
• Optimize file organization: Distribute files across multiple subdirectories to avoid excessive files in a single directory
• Use more suitable filesystems: Choose filesystems that better support large numbers of small files

Programming Best Practices

To prevent similar issues, consider the following when writing file-intensive Python programs:

import os
import errno

def safe_file_write(filepath, content, max_retries=3):
    """Safe file writing function with error handling"""
    for attempt in range(max_retries):
        try:
            # Ensure directory exists
            os.makedirs(os.path.dirname(filepath), exist_ok=True)
            
            with open(filepath, 'w') as f:
                f.write(content)
            return True
            
        except IOError as e:
            if e.errno == errno.ENOSPC:
                print(f"Disk space exhausted, attempt {attempt + 1}/{max_retries}")
                if attempt == max_retries - 1:
                    raise
            else:
                raise
    return False

# Use subdirectories to distribute files
def get_file_path(base_dir, count, files_per_dir=1000):
    """Distribute files across multiple subdirectories"""
    subdir = str(count // files_per_dir)
    filename = str(count % files_per_dir) + '.html'
    return os.path.join(base_dir, subdir, filename)

System Monitoring and Prevention

Regular monitoring of filesystem status can prevent such problems:

• Use df -h and df -i to monitor disk space and inode usage
• Set disk usage alert thresholds
• Regularly check filesystem integrity
• Implement file cleanup strategies to remove unnecessary old files

Conclusion

The IOError: [Errno 28] No space left on device error typically represents more than simple disk space issues, indicating filesystem-level limitations or corruption. Drive formatting provides a fundamental solution, while combining reasonable programming practices with system monitoring can effectively prevent the occurrence of such problems.