openstack - Openstack Victoria SRIOV 没有所需的 PCI 设备
问题描述
我已按照 Openstack Victoria 文档配置 SRIOV ( https://docs.openstack.org/neutron/victoria/admin/config-sriov.html )，我认为自从通过 SRIOV 网络创建端口时，我遗漏了一些东西已配置网络提供商 physnet3 未找到 PCI 设备。
我的服务器：
- 控制器：host02
- 计算：hots09
我的设置：
compute host09:
cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.4.0-72-generic root=/dev/mapper/vg0-lv--root ro isolcpus=42-55 maybe-ubiquity intel_iommu=on iommu=pt hugepagesz=1G hugepages=32 default_hugepagesz=1G transparent_hugepage=never
lspci -v | grep Mel
02:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
86:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
86:00.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:00.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:00.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:00.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:00.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:00.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:00.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:01.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:02.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.1 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.2 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.3 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.4 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.5 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.6 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:03.7 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
86:04.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
cat /etc/nova/nova.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
my_ip =
state_path = /var/lib/nova
enabled_apis = osapi_compute,metadata
log_dir = /var/log/nova
transport_url = rabbit://
use_neutron = True
linuxnet_interface_driver = nova.network.linux_net.LinuxBridgeInterfaceDriver
firewall_driver = nova.virt.firewall.NoopFirewallDriver
vif_plugging_is_fatal = True
vif_plugging_timeout = 300
cpu_allocation_ratio = 3.0
reserved_huge_pages = node:0,size:1GB,count:1
reserved_huge_pages = node:1,size:1GB,count:1
vcpu_pin_set = 42-55
[filter_scheduler]
available_filters=nova.scheduler.filters.all_filters
enabled_filters=AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
[api]
auth_strategy = keystone
[vnc]
enabled = True
server_listen = 0.0.0.0
server_proxyclient_address = $my_ip
novncproxy_base_url = http://
[glance]
api_servers = http://
[oslo_concurrency]
lock_path = $state_path/tmp
[keystone_authtoken]
www_authenticate_uri = http://
auth_url = http://
memcached_servers =
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username =
password =
[placement]
auth_url = http://
os_region_name = RegionOne
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username =
password =
[wsgi]
api_paste_config = /etc/nova/api-paste.ini
[neutron]
auth_url = http://
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username =
password =
service_metadata_proxy = True
metadata_proxy_shared_secret =
[cinder]
os_region_name = RegionOne
[pci]
passthrough_whitelist = { "devname": "enp134s0", "physical_network": "physnet3", "trusted":"true", "devname": "enp134s0d1", "physical_network": "physnet4", "trusted":"true"}
cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
path_mtu = 1550
physical_network_mtus = physnet1:1550
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge
extension_drivers = port_security
path_mtu = 1550
segment_mtu = 1550
physical_network_mtus = physnet1:1500
[ml2_type_flat]
flat_networks = *
[ml2_type_geneve]
[ml2_type_gre]
[ml2_type_vlan]
network_vlan_ranges = physnet2:1500:1520
[ml2_type_vxlan]
vni_ranges = 1:1000
[ovs_driver]
[securitygroup]
[sriov_driver]
cat /etc/neutron/plugins/ml2/sriov_agent.ini | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
[agent]
[sriov_nic]
physical_device_mappings = physnet3:enp134s0,physnet4:enp134s0d1
exclude_devices = enp134s0:86:02.1;86:02.2;86:02.3;86:02.4;86:02.5;86:02.6;86:02.7;86:03.0;86:03.1;86:03.2;86:03.3;86:03.4;86:03.5;86:03.6;86:03.7;86:04.0,enp134s0d1:86:00.1;86:00.2;86:00.3;86:00.4;86:00.5;86:00.6;86:00.7;86:01.0;86:01.1;86:01.2;86:01.3;86:01.4;86:01.5;86:01.6;86:01.7;86:02.0
[securitygroup]
firewall_driver = neutron.agent.firewall.NoopFirewallDriver
cat /etc/neutron/neutron.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
core_plugin = ml2
service_plugins = router
auth_strategy = keystone
state_path = /var/lib/neutron
allow_overlapping_ips = True
transport_url = rabbit://
global_physnet_mtu = 1550
[agent]
root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
www_authenticate_uri = http://
auth_url = http://
memcached_servers =
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username =
password =
[oslo_concurrency]
lock_path = $state_path/lock
controller host02:
openstack network agent list --host host09
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 0ee3ac1e-8ea2-4da9-8143-3a71e4de130c | Metadata agent | host09 | None | :-) | UP | neutron-metadata-agent |
| 32b5a152-92eb-4a37-b513-8b6cbaf7ec31 | Linux bridge agent | host09 | None | :-) | UP | neutron-linuxbridge-agent |
| 89e0a1b1-ecfe-4af2-9c4a-9f355eecd38a | NIC Switch agent | host09 | None | :-) | UP | neutron-sriov-nic-agent |
| b8e51a2e-de07-4d62-b6b4-cbe6d545f8cf | DHCP agent | host09 | nova | :-) | UP | neutron-dhcp-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
openstack network agent list --host host02
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 59a0f618-a66f-44e6-9763-5d29f071d6da | Linux bridge agent | host02 | None | :-) | UP | neutron-linuxbridge-agent |
| de9daf1c-6325-4cff-ab0e-dd1be29b57e8 | L3 agent | host02 | nova | :-) | UP | neutron-l3-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
cat /etc/neutron/neutron.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
core_plugin = ml2
service_plugins = router
auth_strategy = keystone
state_path = /var/lib/neutron
dhcp_agent_notification = True
allow_overlapping_ips = True
notify_nova_on_port_status_changes = True
notify_nova_on_port_data_changes = True
transport_url = rabbit://
global_physnet_mtu = 1550
[agent]
root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
[keystone_authtoken]
www_authenticate_uri = http://
auth_url = http://
memcached_servers =
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username =
password =
[database]
connection = mysql+pymysql://neutron:
[nova]
auth_url = http://
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = nova
password =
[oslo_concurrency]
lock_path = $state_path/tmp
cat /etc/neutron/plugins/ml2/ml2_conf.ini | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
path_mtu = 1550
physical_network_mtus = physnet1:1550
[ml2]
type_drivers = flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge,sriovnicswitch
extension_drivers = port_security
path_mtu = 1550
segment_mtu = 1550
physical_network_mtus = physnet1:1500
[ml2_type_flat]
flat_networks = *
[ml2_type_geneve]
[ml2_type_gre]
[ml2_type_vlan]
network_vlan_ranges = physnet2:1500:1520,physnet3:1504:1506,physnet4:1504:1506
[ml2_type_vxlan]
vni_ranges = 1:1000
[ovs_driver]
[securitygroup]
[sriov_driver]
cat /etc/nova/nova.conf | grep -v "\#" | sed '/^[[:space:]]*$/d'
[DEFAULT]
my_ip =
state_path = /var/lib/nova
enabled_apis = osapi_compute,metadata
log_dir = /var/log/nova
transport_url = rabbit://
use_neutron = True
linuxnet_interface_driver = nova.network.linux_net.LinuxBridgeInterfaceDriver
firewall_driver = nova.virt.firewall.NoopFirewallDriver
vif_plugging_is_fatal = True
vif_plugging_timeout = 300
debug=true
[filter_scheduler]
available_filters=nova.scheduler.filters.all_filters
enabled_filters=AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAffinityFilter,PciPassthroughFilter,NUMATopologyFilter,AggregateInstanceExtraSpecsFilter
[api]
auth_strategy = keystone
[glance]
api_servers = http://
[oslo_concurrency]
lock_path = $state_path/tmp
[api_database]
connection = mysql+pymysql://
[database]
connection = mysql+pymysql://
[keystone_authtoken]
www_authenticate_uri = http://
auth_url = http://
memcached_servers =
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username =
password =
[placement]
auth_url = http://
os_region_name = RegionOne
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username =
password =
[wsgi]
api_paste_config = /etc/nova/api-paste.ini
[neutron]
auth_url = http://
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username =
password =
service_metadata_proxy = True
metadata_proxy_shared_secret =
openstack network show 61e37950-5944-44e1-81df-5aa3a94a016f
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2021-04-19T15:52:42Z |
| description | |
| dns_domain | None |
| id | 61e37950-5944-44e1-81df-5aa3a94a016f |
| ipv4_address_scope | None |
| ipv6_address_scope | None |
| is_default | None |
| is_vlan_transparent | None |
| mtu | 1550 |
| name | sriov-net-1504 |
| port_security_enabled | True |
| project_id | 2467cb8a053d40f0a0a8af43d89d1c32 |
| provider:network_type | vlan |
| provider:physical_network | physnet3 |
| provider:segmentation_id | 1504 |
| qos_policy_id | None |
| revision_number | 2 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | e5dd4fa3-abe7-482c-8544-b73836c547cf |
| tags | |
| updated_at | 2021-04-19T15:59:34Z |
+---------------------------+--------------------------------------+
nova-scheduler.log
2021-04-20 02:26:08.344 450488 DEBUG nova.scheduler.filters.pci_passthrough_filter [req-dec3be12-9274-4733-94a1-7e29dc4b6de2 3b1d6ae009c94f99800bb1a53cedba4a 2467cb8a053d40f0a0a8af43d89d1c32 - default default] (
magma-host09, magma-host09) ram: 171084MB disk: 13312MB io_ops: 0 instances: 5 doesn't have the required PCI devices (InstancePCIRequests(instance_uuid=<?>,requests=[InstancePCIRequest])) host_passes /usr/lib/py
thon3/dist-packages/nova/scheduler/filters/pci_passthrough_filter.py:52
正如您在 nova 调度程序的日志中看到的那样，它没有找到 PCI 设备
没有所需的 PCI 设备
我将非常感谢您的帮助，问候。
解决方案
我发现了错误，我在 nova compute 上的 passthrough_whitelist 是错误的：
wrong:
passthrough_whitelist = { "devname": "enp134s0", "physical_network": "physnet3", "trusted":"true", "devname": "enp134s0d1", "physical_network": "physnet4", "trusted":"true"}
Fixed multiple values:
passthrough_whitelist = { "devname": "enp134s0", "physical_network": "physnet3", "trusted":"true" }
passthrough_whitelist = { "devname": "enp134s0d1", "physical_network": "physnet4", "trusted":"true" }
推荐阅读
- sql - 如何在 Oracle SQL 中将两个查询合二为一
- google-app-maker - Google App Maker：如何在绑定到计算模型的表上实现分页
- python - Tkinter：在其他图像之上显示图像
- postgresql - 有没有办法在 Debezium Postgres 连接器中启用支持以捕获复合类型列？
- elixir - Elixir：当前没有与给定名称关联的进程
- scala - 如何避免对 Spark 数据帧的嵌套调用 - 这不起作用
- spring - Spring可为空的注释生成未知的枚举常量警告
- security - 将 CSRF Token 放在 GET 请求 URL 中是否有风险？
- linux - 执行脚本后始终清理 PowerShell 窗口
- django - Django：如何从数据库中获取倒数第二个对象？