Efficient Byte Array Concatenation in C#: Performance Analysis and Best Practices

Introduction

Concatenating multiple byte arrays is a common requirement in C# programming, particularly in scenarios involving network data processing, file I/O operations, and memory management. Selecting the appropriate concatenation method not only impacts code readability but directly influences application performance. This article systematically analyzes the efficiency characteristics of various byte array concatenation methods based on comprehensive performance test data.

Performance Testing Environment and Methodology

To accurately evaluate the performance differences among various concatenation methods, we designed multiple test rounds using byte arrays of different scales for benchmarking. The testing environment included:

• Test loops executed 1 million times (small arrays) or 4,000 times (large arrays)
• Array sizes ranging from 10 bytes to 1 million bytes
• Measurement metrics including creation time and subsequent iteration time

Major Concatenation Methods Comparison

System.Buffer.BlockCopy Method

System.Buffer.BlockCopy is a high-efficiency copying method specifically designed for primitive types, including bytes. Compared to System.Array.Copy, it demonstrates significant performance advantages when handling byte arrays.

Basic usage example:

byte[] rv = new byte[a1.Length + a2.Length + a3.Length];
System.Buffer.BlockCopy(a1, 0, rv, 0, a1.Length);
System.Buffer.BlockCopy(a2, 0, rv, a1.Length, a2.Length);
System.Buffer.BlockCopy(a3, 0, rv, a1.Length + a2.Length, a3.Length);

For arbitrary numbers of arrays, a generic method can be written:

private byte[] Combine(params byte[][] arrays)
{
    byte[] rv = new byte[arrays.Sum(a => a.Length)];
    int offset = 0;
    foreach (byte[] array in arrays) {
        System.Buffer.BlockCopy(array, 0, rv, offset, array.Length);
        offset += array.Length;
    }
    return rv;
}

LINQ Concat Method

The LINQ Concat method provides a concise and elegant approach to array concatenation, particularly suitable for scenarios requiring deferred execution.

Usage example:

IEnumerable<byte> rv = a1.Concat(a2).Concat(a3);

This method returns an IEnumerable<byte> sequence that supports deferred execution, making it highly efficient when only traversal of results is needed without immediate creation of a complete array.

C# Yield Operator

Using the yield operator enables creation of custom iterators, achieving deferred execution effects similar to LINQ Concat.

Performance Test Results Analysis

Small Array Performance Comparison

For small arrays of 10 bytes, test results show:

• System.Array.Copy: 0.2187556 seconds
• System.Buffer.BlockCopy: 0.1406286 seconds
• Yield operator: 0.0781270 seconds
• LINQ Concat: 0.0781270 seconds

Medium-Scale Array Performance

When array size increases to 1,000 bytes:

• System.Array.Copy: 1.0781457 seconds
• System.Buffer.BlockCopy: 1.0156445 seconds
• Yield operator: 0.0625012 seconds
• LINQ Concat: 0.0781265 seconds

Large-Scale Array Performance

For large arrays of 1 million bytes:

• System.Array.Copy: 13.4533833 seconds
• System.Buffer.BlockCopy: 13.1096267 seconds
• Yield operator: 0 seconds (iterator creation only)
• LINQ Concat: 0 seconds (iterator creation only)

Comprehensive Efficiency of Creation and Usage

An important finding from performance testing is that comparing creation efficiency alone can be misleading. When considering subsequent iteration usage of the resulting data structure, the situation changes significantly.

In comprehensive tests including iteration:

• System.Array.Copy: 78.20550510 seconds
• System.Buffer.BlockCopy: 77.89261900 seconds
• Yield operator: 551.7150161 seconds
• LINQ Concat: 448.1804799 seconds

This indicates that while deferred execution methods are highly efficient during the creation phase, they may incur significant performance overhead during subsequent iteration usage.

Semantic Differences and Application Scenarios

Immediate Copying vs Deferred Execution

Different concatenation methods exhibit important semantic differences:

• Immediate copying methods (BlockCopy, Array.Copy) create independent copies of data, where subsequent modifications to source arrays do not affect concatenation results
• Deferred execution methods (LINQ Concat, yield) dynamically access source arrays during iteration, meaning that if source arrays are modified after creation, these changes are reflected in iteration results

Memory Considerations

Immediate copying methods require allocation of contiguous memory space equal to the total concatenated size, while deferred execution methods access data on-demand during iteration, offering more flexible memory usage.

Best Practice Recommendations

When Byte Arrays Are Required

If the application requires actual byte arrays:

• Prioritize System.Buffer.BlockCopy, which offers optimal performance for primitive types
• Use specific BlockCopy calls for fixed numbers of arrays
• Use generic Combine methods for variable numbers of arrays

Suitable Scenarios for Deferred Execution

If IEnumerable<byte> is acceptable:

• Prefer LINQ's Concat method, which balances conciseness and performance effectively
• The yield operator, while slightly better performing, offers poorer code readability
• Particularly suitable for scenarios requiring only single traversal or handling extremely large data volumes

Performance Optimization Considerations

In practical applications, selecting concatenation methods should consider:

• Data scale: Small data suits any method, large data requires careful selection
• Usage patterns: Single use favors deferred execution, multiple uses favor immediate copying
• Memory constraints: Large array concatenation may create significant memory pressure
• Thread safety: Deferred execution methods require additional attention in multi-threaded environments

Conclusion

The optimal method for byte array concatenation in C# depends on specific application requirements. For scenarios requiring immediate byte arrays, System.Buffer.BlockCopy provides the best performance. For scenarios accepting deferred execution, LINQ's Concat method offers good performance while maintaining code conciseness. Developers should select the most appropriate concatenation strategy based on data scale, usage patterns, and performance requirements.

Understanding the differences in semantic and performance characteristics among various methods is crucial to avoid one-sided optimization based solely on creation time. In practical projects, we recommend validating performance in specific scenarios through benchmarking to ensure selected methods meet functional requirements while providing acceptable performance characteristics.