c++ - 将 boost 的 scoped_allocator_adaptor 用于共享内存容器

问题描述

我正在编写一个 C++17 应用程序，我需要在共享内存中管理一个 STL 或boost::collections等效数据结构。

我不确定创建和更新共享数据结构的最简单语法（避免到处传递分配器）。

我已经搜索了一段时间，但除了一个简单的 String->String 映射之外，很难找到专注于自定义数据结构或 POD 结构的示例。（我怀疑与 POD 结构相关的分配器会相当简单，因为它们可以从连续的内存中分配，因此可以使用简单的 char 分配器 - 相当于Shared::Alloc<char>下面）。

据我了解，在共享内存中管理数据结构集合的关键在于正确选择有状态的分配器以及让该分配器与其嵌套的子级共享的能力。

例如，假设我map<Shared::String, vector<Shared::String>> 在共享内存中有一个，它的魔力scoped_allocator_adaptor会起作用。

除了上面的简单示例map<SHMString, vector<String>>之外，我真的很想管理一个map<SHMString, vector<UserStruct>>whereUserStruct可以是 POD 结构或包含一个String或List字符串的结构。

我从以下内容开始，作为我在 SO 中找到的另一个答案的有用起点：

namespace bip = boost::interprocess;

namespace Shared {
    using Segment = bip::managed_shared_memory;

    template <typename T>
        using Alloc   = bip::allocator<T, Segment::segment_manager>;
    using Scoped  = boost::container::scoped_allocator_adaptor<Alloc<char>>;

    using String  = boost::container::basic_string<char, std::char_traits<char>, Scoped>;
    using KeyType = String;
}

看起来Shared:Scoped分配器适配器是将分配器从顶级容器传播到其子级的关键。我不确定这在应用于增强容器与标准容器时是否不同。

我正在寻找一个示例，以及有关如何以允许我将其传播scoped_allocator_adaptor到我的 POD 或自定义结构的方式构造这些对象的说明。

标签： c++boostshared-memoryboost-interprocess

解决方案

为星星射击，我们是不是 :) 无痛分配器传播是圣杯。

看起来 Shared:Scoped 分配器适配器是将分配器从顶级容器传播到其子容器的关键。

确实

我不确定这在应用于增强容器与标准容器时是否不同。

在我的理解中，现代 C++ 标准库应该支持相同的，但在实践中，我的经验表明它经常与 Boost Container 容器一起使用。（YMMV 和标准库实现可能/将会赶上）

该怎么办

我想你会想了解uses_allocator协议：https ://en.cppreference.com/w/cpp/memory/uses_allocator

我想这真的回答了你所有的问题。如果可以的话，我会尝试提供一个快速示例。

演示

到目前为止，我已经采用了以下两种方法：

struct MyStruct {
    String data;

    using allocator_type = Alloc<char>;

    MyStruct(MyStruct const& rhs, allocator_type = {}) : data(rhs.data) {}
    template <typename I, typename = std::enable_if_t<not std::is_same_v<MyStruct, I>, void> >
    MyStruct(I&& init, allocator_type a)
     : data(std::forward<I>(init), a)
    { }
};

这允许：

Shared::Segment mf(bip::open_or_create, "test.bin", 10<<20);

auto& db = *mf.find_or_construct<Shared::Database>("db")(mf.get_segment_manager());

db.emplace_back("one");
db.emplace_back("two");
db.emplace_back("three");

稍微复杂/通用（？）的方法也有效：

    MyStruct(std::allocator_arg_t, allocator_type, MyStruct const& rhs) : data(rhs.data) {}

    template <
        typename I,
        typename A = Alloc<char>,
        typename = std::enable_if_t<not std::is_same_v<MyStruct, I>, void> >
    MyStruct(std::allocator_arg_t, A alloc, I&& init)
     : data(std::forward<I>(init), alloc.get_segment_manager())
    { }

看来，对于当前的用例，内部 typedefallocator_type足以表明MyStruct支持分配器构造，从而使专业化uses_allocator<MyStruct, ...>变得多余。

完整列表

Live On Coliru

#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <iostream>

namespace bip = boost::interprocess;

namespace Shared {
    using Segment = bip::managed_mapped_file;
    using SMgr = Segment::segment_manager;

    template <typename T> using Alloc = boost::container::scoped_allocator_adaptor<
            bip::allocator<T, SMgr>
        >;

    template <typename T> using Vec = boost::container::vector<T, Alloc<T> >;

    using String = bip::basic_string<char, std::char_traits<char>, Alloc<char> >;

    struct MyStruct {
        String data;

        using allocator_type = Alloc<char>;

#if 1 // one approach
        MyStruct(std::allocator_arg_t, allocator_type, MyStruct const& rhs) : data(rhs.data) {}

        template <
            typename I,
            typename A = Alloc<char>,
            typename = std::enable_if_t<not std::is_same_v<MyStruct, I>, void> >
        MyStruct(std::allocator_arg_t, A alloc, I&& init)
         : data(std::forward<I>(init), alloc.get_segment_manager())
        { }
#else // the simpler(?) approach
        MyStruct(MyStruct const& rhs, allocator_type = {}) : data(rhs.data) {}
        template <typename I, typename = std::enable_if_t<not std::is_same_v<MyStruct, I>, void> >
        MyStruct(I&& init, allocator_type a)
         : data(std::forward<I>(init), a)
        { }
#endif
    };

    using Database = Vec<MyStruct>;
}

namespace std {
    // this appears optional for the current use case
    template <typename T> struct uses_allocator<Shared::MyStruct, T> : std::true_type {};
}

int main() {
    Shared::Segment mf(bip::open_or_create, "test.bin", 10<<20);

    auto& db = *mf.find_or_construct<Shared::Database>("db")(mf.get_segment_manager());

    db.emplace_back("one");
    db.emplace_back("two");
    db.emplace_back("three");

    std::cout << "db has " << db.size() << " elements:";

    for (auto& el : db) {
        std::cout << " " << el.data;
    }

    std::cout << std::endl;
}

调用它三遍：

db has 3 elements: one two three
db has 6 elements: one two three one two three
db has 9 elements: one two three one two three one two three

更新：更复杂

针对评论，让我们通过两种方式使其更复杂：

struct 构造函数将采用各种参数来初始化各种成员，其中一些将使用分配器。
我们想将它存储在一个 Map 中，并且一些涉及 map 的使用模式对于范围分配器的支持是令人讨厌map[k]=v的（具有默认构造要求的位置、更新分配）
std::initalizer_list<>不会在通用转发包装器中推断出来:(

定义结构：

struct MyPodStruct {
    using allocator_type = ScopedAlloc<char>;

    int a = 0; // simplify default constructor using NSMI
    int b = 0;
    Vec<uint8_t> data;

    explicit MyPodStruct(allocator_type alloc) : data(alloc) {}
    //MyPodStruct(MyPodStruct const&) = default;
    //MyPodStruct(MyPodStruct&&) = default;
    //MyPodStruct& operator=(MyPodStruct const&) = default;
    //MyPodStruct& operator=(MyPodStruct&&) = default;

    MyPodStruct(std::allocator_arg_t, allocator_type, MyPodStruct&& rhs) : MyPodStruct(std::move(rhs)) {}
    MyPodStruct(std::allocator_arg_t, allocator_type, MyPodStruct const& rhs) : MyPodStruct(rhs) {}

    template <typename I, typename A = Alloc<char>>
        MyPodStruct(std::allocator_arg_t, A alloc, int a, int b, I&& init)
         : MyPodStruct(a, b, Vec<uint8_t>(std::forward<I>(init), alloc)) { }

  private:
    explicit MyPodStruct(int a, int b, Vec<uint8_t> data) : a(a), b(b), data(std::move(data)) {}
};

它解决了“默认构造”（在使用分配器机制下）以及采用多个参数的各种构造函数。并不是说不再需要 SFINAE 来消除uses-allocator copy-constructor 的歧义，因为参数的数量不同。

现在，使用它比上面更复杂。具体来说，由于要转发多个构造函数参数，我们需要另外一点“构造协议”：std::piece_wise_construct_t.

内联评论讨论了 QoL/QoI 问题和陷阱：

int main() {
    using Shared::MyPodStruct;
    Shared::Segment mf(bip::open_or_create, "test.bin", 10<<10); // smaller for Coliru
    auto mgr = mf.get_segment_manager();

    auto& db = *mf.find_or_construct<Shared::Database>("complex")(mgr);

    // Issues with brace-enclosed initializer list
    using Bytes = std::initializer_list<uint8_t>;

    // More magic: piecewise construction protocol :)
    static constexpr std::piecewise_construct_t pw{};
    using std::forward_as_tuple;
    db.emplace(pw, forward_as_tuple("one"), forward_as_tuple(1,2, Bytes {1,2}));
    db.emplace(pw, forward_as_tuple("two"), forward_as_tuple(2,3, Bytes {4}));
    db.emplace(pw, forward_as_tuple("three"), forward_as_tuple(3,4, Bytes {5,8}));

    std::cout << "\n=== Before updates\n" << db << std::endl;

    // Clumsy:
    db[Shared::String("one", mgr)] = MyPodStruct{std::allocator_arg, mgr, 1,20, Bytes {7,8,9}};

    // As efficient or better, and less clumsy:
    auto insert_or_update = [&db](auto&& key, auto&&... initializers) -> MyPodStruct& {
        // Be careful not to move twice: https://en.cppreference.com/w/cpp/container/map/emplace
        // > The element may be constructed even if there already is an element
        // > with the key in the container, in which case the newly constructed
        // > element will be destroyed immediately.
        if (auto insertion = db.emplace(pw, forward_as_tuple(key), std::tie(initializers...)); insertion.second) {
            return insertion.first->second;
        } else {
            return insertion.first->second = MyPodStruct(
                std::allocator_arg, 
                db.get_allocator(),
                std::forward<decltype(initializers)>(initializers)...); // forwarding ok here
        }
    };

    insert_or_update("two", 2,30, Bytes{});
    insert_or_update("nine", 9,100, Bytes{5,6});

    // partial updates:
    db.at(Shared::String("nine", mgr)).data.push_back(42);

    // For more efficient key lookups in the case of unlikely insertion, use
    // heterogeneous comparer, see https://stackoverflow.com/a/27330042/85371

    std::cout << "\n=== After updates\n" << db << std::endl;
}

哪个打印Live On Coliru

=== Before updates
db has 3 elements: {one: 1,2, [1,2,]} {three: 3,4, [5,8,]} {two: 2,3, [4,]}

=== After updates
db has 4 elements: {nine: 9,100, [5,6,42,]} {one: 1,20, [7,8,9,]} {three: 3,4, [5,8,]} {two: 2,30, []}

完整列表

对于保护：Live On Coliru

#include <boost/interprocess/containers/map.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <iostream>

namespace bip = boost::interprocess;

namespace Shared {
    using Segment = bip::managed_mapped_file;
    using SMgr = Segment::segment_manager;

    template <typename T> using Alloc = bip::allocator<T, SMgr>;
    template <typename T> using ScopedAlloc = boost::container::scoped_allocator_adaptor<Alloc<T> >;

    using String = bip::basic_string<char, std::char_traits<char>, Alloc<char> >;

    using boost::interprocess::map;

    template <typename T> using Vec = 
        boost::container::vector<T, ScopedAlloc<T>>;

    template <typename K, typename T> using Map = 
        map<K, T, std::less<K>, ScopedAlloc<typename map<K, T>::value_type>>;

    struct MyPodStruct {
        using allocator_type = ScopedAlloc<char>;

        int a = 0; // simplify default constructor using NSMI
        int b = 0;
        Vec<uint8_t> data;

        explicit MyPodStruct(allocator_type alloc) : data(alloc) {}
        //MyPodStruct(MyPodStruct const&) = default;
        //MyPodStruct(MyPodStruct&&) = default;
        //MyPodStruct& operator=(MyPodStruct const&) = default;
        //MyPodStruct& operator=(MyPodStruct&&) = default;

        MyPodStruct(std::allocator_arg_t, allocator_type, MyPodStruct&& rhs) : MyPodStruct(std::move(rhs)) {}
        MyPodStruct(std::allocator_arg_t, allocator_type, MyPodStruct const& rhs) : MyPodStruct(rhs) {}

        template <typename I, typename A = Alloc<char>>
            MyPodStruct(std::allocator_arg_t, A alloc, int a, int b, I&& init)
             : MyPodStruct(a, b, Vec<uint8_t>(std::forward<I>(init), alloc)) { }

      private:
        explicit MyPodStruct(int a, int b, Vec<uint8_t> data) : a(a), b(b), data(std::move(data)) {}
    };    

    using Database = Map<String, MyPodStruct>;

    static inline std::ostream& operator<<(std::ostream& os, Database const& db) {
        os << "db has " << db.size() << " elements:";

        for (auto& [k,v] : db) {
            os << " {" << k << ": " << v.a << "," << v.b << ", [";
            for (unsigned i : v.data)
                os << i << ",";
            os << "]}";
        }

        return os;
    }
}

int main() {
    using Shared::MyPodStruct;
    Shared::Segment mf(bip::open_or_create, "test.bin", 10<<10); // smaller for Coliru
    auto mgr = mf.get_segment_manager();

    auto& db = *mf.find_or_construct<Shared::Database>("complex")(mgr);

    // Issues with brace-enclosed initializer list
    using Bytes = std::initializer_list<uint8_t>;

    // More magic: piecewise construction protocol :)
    static constexpr std::piecewise_construct_t pw{};
    using std::forward_as_tuple;
    db.emplace(pw, forward_as_tuple("one"), forward_as_tuple(1,2, Bytes {1,2}));
    db.emplace(pw, forward_as_tuple("two"), forward_as_tuple(2,3, Bytes {4}));
    db.emplace(pw, forward_as_tuple("three"), forward_as_tuple(3,4, Bytes {5,8}));

    std::cout << "\n=== Before updates\n" << db << std::endl;

    // Clumsy:
    db[Shared::String("one", mgr)] = MyPodStruct{std::allocator_arg, mgr, 1,20, Bytes {7,8,9}};

    // As efficient or better, and less clumsy:
    auto insert_or_update = [&db](auto&& key, auto&&... initializers) -> MyPodStruct& {
        // Be careful not to move twice: https://en.cppreference.com/w/cpp/container/map/emplace
        // > The element may be constructed even if there already is an element
        // > with the key in the container, in which case the newly constructed
        // > element will be destroyed immediately.
        if (auto insertion = db.emplace(pw, forward_as_tuple(key), std::tie(initializers...)); insertion.second) {
            return insertion.first->second;
        } else {
            return insertion.first->second = MyPodStruct(
                std::allocator_arg, 
                db.get_allocator(),
                std::forward<decltype(initializers)>(initializers)...); // forwarding ok here
        }
    };

    insert_or_update("two", 2,30, Bytes{});
    insert_or_update("nine", 9,100, Bytes{5,6});

    // partial updates:
    db.at(Shared::String("nine", mgr)).data.push_back(42);

    // For more efficient key lookups in the case of unlikely insertion, use
    // heterogeneous comparer, see https://stackoverflow.com/a/27330042/85371

    std::cout << "\n=== After updates\n" << db << std::endl;
}

c++ - 将 boost 的 scoped_allocator_adaptor 用于共享内存容器

问题描述

解决方案

该怎么办

演示

完整列表

更新：更复杂

完整列表

推荐阅读