Parsing Complex Text Files with C#: From Manual Handling to Automated Solutions

When dealing with large-scale text data, manual operations are not only inefficient but also prone to errors. This article uses a complex text file with 5000 lines, each delimited by tabs, as an example to demonstrate how to implement automated parsing in C#, extracting key information such as the second integer and specific file paths.

Problem Background and Challenges

The original text file contains multiple fields per line, separated by tabs, for example: 1\t1\tITEM_ETC_GOLD_01\t골드(소)\txxx xxx xxx_TT_DESC\t0\t0\t3\t3\t5\t0\t180000\t3\t0\t1\t0\t0\t255\t1\t1\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t-1\t0\t-1\t0\t-1\t0\t-1\t0\t-1\t0\t0\t0\t0\t0\t0\t0\t100\t0\t0\t0\txxx\titem\etc\drop_ch_money_small.bsr\txxx\txxx\txxx\t0\t2\t0\t0\t1\t0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t0.0\t1\t표현할 골드의 양(param1이상)\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t-1\txxx\t0\t0. The goal is to extract the second integer (e.g., 1, 4, 5) and the path string starting with item\ and ending with .ddj from each line. Manual processing of such data is impractical, necessitating a programming solution.

String Splitting-Based Parsing Method

This method uses tabs as delimiters to split each line into a string array, then directly accesses and searches for the required data. The implementation steps are as follows:

1. Use StreamReader to read the file line by line.
2. Apply the Split('\t') method to each line to generate a field array.
3. Retrieve the second field by index and parse it as an integer.
4. Iterate through the array, using conditions StartsWith("item\\") and EndsWith(".ddj") to locate the path.

Example code:

using System.IO;

class Program
{
    static void Main()
    {
        using (StreamReader reader = File.OpenText("filename.txt"))
        {
            string line;
            while ((line = reader.ReadLine()) != null)
            {
                string[] items = line.Split('\t');
                if (items.Length > 1)
                {
                    int myInteger = int.Parse(items[1]);
                    string path = null;
                    foreach (string item in items)
                    {
                        if (item.StartsWith("item\\") && item.EndsWith(".ddj"))
                        {
                            path = item;
                            break;
                        }
                    }
                    Console.WriteLine($"Integer: {myInteger}, Path: {path}");
                }
            }
        }
    }
}

This method is straightforward and suitable for scenarios with clear delimiters and relatively fixed data structures. The time complexity is O(n*m), where n is the number of lines and m is the number of fields per line, which is acceptable for 5000 lines of data.

Regular Expression-Based Parsing Method

Regular expressions offer more flexible matching patterns, especially for complex or dynamic data formats. The following regular expression can precisely extract the target data: ^\d+\t(\d+)\t.+?\t(item\\[^\t]+\.ddj).

• ^\d+\t: Matches the digit at the start of the line and a tab.
• (\d+): Captures the second integer.
• \t.+?\t: Lazily matches the middle content up to the next tab.
• (item\\[^\t]+\.ddj): Captures the path starting with item\, containing no tabs, and ending with .ddj.

Example code:

using System.IO;
using System.Text.RegularExpressions;

class Program
{
    static void Main()
    {
        Regex parts = new Regex(@"^\d+\t(\d+)\t.+?\t(item\\[^\t]+\.ddj)");
        using (StreamReader reader = File.OpenText("filename.txt"))
        {
            string line;
            while ((line = reader.ReadLine()) != null)
            {
                Match match = parts.Match(line);
                if (match.Success)
                {
                    int number = int.Parse(match.Groups[1].Value);
                    string path = match.Groups[2].Value;
                    Console.WriteLine($"Integer: {number}, Path: {path}");
                }
            }
        }
    }
}

The regular expression method is more advantageous when patterns are complex or data format validation is needed, though it may be slightly slower than string splitting and requires attention to special character escaping.

Method Comparison and Best Practices

Both methods have their pros and cons: string splitting is easy to implement and debug, suitable for simple delimiter scenarios; regular expressions are more powerful for handling irregular data. In practice, it is recommended to:

1. For files with fixed delimiters, prioritize string splitting for better performance.
2. Use regular expressions if data includes variable patterns or requires complex validation.
3. Incorporate error handling (e.g., try-catch for integer parsing) to enhance robustness.
4. Use using statements to ensure resource release and avoid memory leaks.

Extended applications may include storing results in a database, generating reports, or integrating into larger systems.

Conclusion

Through C#'s string operations and regular expressions, developers can efficiently parse complex text files, avoiding the risks of manual handling. The methods demonstrated in this article not only solve specific problems but also provide a general framework for similar data processing tasks. In real-world projects, selecting the appropriate method based on data characteristics and requirements can significantly improve development efficiency and code quality.