时间戳

首页 > 解决方案 > 从套接字接收一个大文件会淹没内存，我是否有内存泄漏？

c++ - 从套接字接收一个大文件会淹没内存，我是否有内存泄漏？

问题描述

我想将一个大文件从一台机器发送到另一台机器。所有字节都发送成功，在另一端也收到了，但是操作大量命中接收端的内存。内存使用量在几秒钟内以每秒约 400MB 的速度上升，然后又稳定下来。我测试的文件大小约为 7 gigs。

这是我用来从套接字读取到文件中的代码：

size_t recv_file(int socket, std::filesystem::path& dest_file) {
    uint64_t bytes_to_read = recv_byte_count(socket);
    char* const file_buffer = new char[MAX_BUFFER_FILE];
    size_t total_received{ 0 };

    try {
        std::fstream fs;
        fs.open(dest_file, std::fstream::in | std::fstream::out | std::fstream::app);

        while (bytes_to_read > 0) {
            const unsigned int actual_bytes_to_read = (bytes_to_read > MAX_BUFFER_FILE) ? MAX_BUFFER_FILE : bytes_to_read;

            size_t received = recv_raw(socket, file_buffer, actual_bytes_to_read, MAX_BUFFER_FILE);

            if (received > 0) {
                fs.write(file_buffer, received);
                bytes_to_read -= received;
                total_received += received;
                std::cout << "Bytes received: " << total_received << std::endl;
                memset(file_buffer, 0, MAX_BUFFER_FILE);


            } else throw WSA_socket_exception(WSAGetLastError(), socket);

        }

        fs.close();
        delete[] file_buffer;
    }
    catch (...) {
        delete[] file_buffer;
        return 0;
    }
    return total_received;
}


size_t recv_raw(const int socket, void* buffer, size_t bytes_to_read, int buffer_size = MAX_BUFFER) {
    char* ptr = static_cast<char*>(buffer);
    size_t received_total{ 0 };

    while (bytes_to_read > 0) {
        const int actual_bytes_to_read{ std::min(buffer_size, static_cast<int>(bytes_to_read)) };

        const int received = recv(socket, ptr, actual_bytes_to_read, 0);
        if (received == 0 || received < -1) throw connection_close_exception(socket);
        if (received > 0) {
            bytes_to_read -= static_cast<size_t>(received);
            ptr += received;
            received_total += static_cast<size_t>(received);
        }
    }
    return received_total;
}

标签： c++socketsnetwork-programming

解决方案

在从接收到最终写入文件的过程中，数据会被多次复制。

tcp 和 file 都可能分配许多临时缓冲区，等待接收到的被调用或文件（部分）刷新到文件系统。Tcp 由定义它可以分配多少的 tcp 设置控制，fs 中可能有类似的设置，否则您可以通过滚动您自己的 std::streambuf 子类来滚动自己的设置。

推荐阅读