Analysis and Solutions for "Resource temporarily unavailable" Error in Socket send() Operations

Error Mechanism Deep Analysis

In Linux socket programming, the "Resource temporarily unavailable" error message corresponds to the system error code EAGAIN. This error indicates that the current operation should normally block and wait, but since the socket is set to non-blocking mode, the system immediately returns an error instead of suspending the process. For the send() system call, this situation mainly occurs when the send buffer is full and the kernel cannot immediately accept more data to be sent.

Non-blocking Mode Configuration Methods

Three main configuration approaches that cause send() operations to return EAGAIN:

fcntl() File Descriptor Setting: Explicitly mark the socket file descriptor as non-blocking mode through fcntl(fd, F_SETFL, O_NONBLOCK). In this mode, all I/O operations return immediately, setting errno to EAGAIN if they cannot complete immediately.

MSG_DONTWAIT Flag: Pass the MSG_DONTWAIT parameter when calling send(), enabling non-blocking behavior only for the current send operation. This approach provides finer-grained control, allowing mixed use of blocking and non-blocking operations on the same socket.

SO_SNDTIMEO Timeout Option: Use setsockopt() to set the SO_SNDTIMEO socket option, specifying the timeout for send operations. When the send cannot complete within the specified time, the system returns an EAGAIN error.

Buffer Management and Flow Control

The socket send buffer is a memory area maintained by the kernel for temporarily storing data to be sent. When an application calls send(), data is first copied to the send buffer, then asynchronously sent by the network protocol stack. The size of the send buffer is limited by system configuration and socket options. When the buffer is full, normal blocking sockets suspend the sending process until buffer space becomes available.

In non-blocking mode, a full buffer condition causes send() to immediately return EAGAIN. The application should then wait for the socket to become writable before retrying the send operation. Changes in buffer status can be monitored through I/O multiplexing mechanisms such as select(), poll(), or epoll().

Practical Case: Kea DHCP Lock File Issue

The Kea DHCP service failure described in the reference article provides a typical EAGAIN error instance. When the Kea process terminates abnormally, it leaves lock files for control sockets (such as /tmp/kea4-ctrl-socket.lock) in the /tmp directory. Subsequent attempts to restart the service fail because the new process cannot acquire these lock files, as filesystem operations also follow similar resource competition principles.

Error logs show: cannot lock socket lockfile, /tmp/kea4-ctrl-socket.lock, : Resource temporarily unavailable. This indicates that the file locking operation encountered an EAGAIN situation, typically because another process (possibly a zombie process) still holds the lock.

Programming Practices and Solutions

Proper handling of EAGAIN errors requires applications to implement appropriate retry mechanisms. Below is a complete example of non-blocking socket sending:

#include <sys/socket.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>

int send_nonblocking(int sockfd, const void *buf, size_t len, int flags) {
    ssize_t sent = 0;
    
    while (sent < len) {
        ssize_t n = send(sockfd, (char*)buf + sent, len - sent, 
                        flags | MSG_DONTWAIT);
        
        if (n > 0) {
            sent += n;
        } else if (n == 0) {
            // Connection closed
            break;
        } else {
            if (errno == EAGAIN || errno == EWOULDBLOCK) {
                // Buffer full, need to wait
                struct timeval timeout = {5, 0}; // 5-second timeout
                fd_set write_fds;
                FD_ZERO(&write_fds);
                FD_SET(sockfd, &write_fds);
                
                int ready = select(sockfd + 1, NULL, &write_fds, NULL, &timeout);
                if (ready <= 0) {
                    // Timeout or error
                    return -1;
                }
                // Socket writable, continue sending
                continue;
            } else {
                // Other errors
                return -1;
            }
        }
    }
    
    return sent;
}

This implementation demonstrates a complete error handling flow: when encountering EAGAIN, use select() to wait for the socket to become writable, then retry the send operation. This approach avoids busy waiting and improves system resource utilization.

System-Level Tuning Recommendations

For high-concurrency network applications, consider the following system-level optimizations:

Adjust Buffer Sizes: Increase send buffers through setsockopt() with the SO_SNDBUF option to reduce the frequency of EAGAIN occurrences. However, note that excessively large buffers increase memory usage and transmission latency.

Use Edge Triggering: Employ EPOLLET edge-triggered mode in epoll to notify applications only when socket state changes, reducing unnecessary system calls.

Implement Backpressure Mechanisms: Implement flow control at the application layer, appropriately reducing send rates when consistently encountering EAGAIN to avoid overwhelming the receiver.

Error Prevention and Debugging

Best practices for preventing EAGAIN errors include: designing application protocols reasonably to avoid burst data transmission; implementing comprehensive error handling and retry logic; using appropriate monitoring tools to track socket buffer status.

For debugging, use netstat -t to view socket send queue lengths, or trace system call sequences with strace to analyze the specific timing and context of send() failures.