django - 在 Django 中将两个不相关的表/模型与相同的主键组合起来
问题描述
我有两个具有相同主键的不相关表。
ip mac
11.11.11.11 48-C0-09-1F-9B-54
33.33.33.33 4E-10-A3-BC-B8-9D
44.44.44.44 CD-00-60-08-56-2A
55.55.55.55 23-CE-D3-B1-39-A6
ip type owner
22.22.22.22 laptop John Doe
33.33.33.33 server XYZ Department
44.44.44.44 VM Mary Smith
66.66.66.66 printer ZWV Department
第一个表每分钟自动刷新一次。我无法更改数据库结构或填充它的脚本。
两个表都具有ip
PRIMARY KEY。
在一个视图中,我想显示一个这样的表格:
ip mac type owner Alert
11.11.11.11 48-C0-09-1F-9B-54 Unauthorized
55.55.55.55 23-CE-D3-B1-39-A6 Unauthorized
22.22.22.22 laptop John Doe Down
66.66.66.66 printer ZWV Department Down
33.33.33.33 4E-10-A3-BC-B8-9D server XYZ Department OK
44.44.44.44 CD-00-60-08-56-2A VM Mary Smith OK
我该如何建模?我应该将两个主键之一作为另一个主键的外键吗?
一旦代码运行起来,就会有很多数据,所以我想确保它足够快。
检索数据的最快方法是什么?
更新:
我尝试使用OneToOneField
第二张桌子。
这有助于我获取两个表中的记录,以及未经授权的设备的记录(第二个表中缺少 IP):
ip mac type owner Alert
11.11.11.11 48-C0-09-1F-9B-54 Unauthorized
55.55.55.55 23-CE-D3-B1-39-A6 Unauthorized
33.33.33.33 4E-10-A3-BC-B8-9D server XYZ Department OK
44.44.44.44 CD-00-60-08-56-2A VM Mary Smith OK
但我无法获取已关闭的设备(第一个表中缺少 IP):
22.22.22.22 laptop John Doe Down
66.66.66.66 printer ZWV Department Down
我在这里寻求帮助,但似乎无法完成OneToOneField
解决方案
大概的概念
您可以使用qs.union:
- 创建 2 个模型,它们之间没有任何关系。不要忘记使用
class Meta: managed = False
- 从第一个模型中选择,用子查询注释并用第二个模型联合:
from django.db import models
from django.db.models import F, OuterRef, Subquery, Value
from django.db.models.functions import Coalesce
# OperationalDevice fields: ip, mac
# AllowedDevice fields: ip, type, owner
USE_EMPTY_STR_AS_DEFAULT = True
null_char_field = models.CharField(null=True)
if USE_EMPTY_STR_AS_DEFAULT:
default_value = ''
else:
default_value = None
# By default Expressions treat strings as "field_name" so if you want to use
# empty string as a second argument for Coalesce, then you should wrap it in
# `Value()`.
# `None` can be used there without wrapping in `Value()`, but in
# `.annotate(type=NoneValue)` it still should be wrapped, so it's easier to
# just "always wrap".
default_value = Value(default_value, output_field=null_char_field)
operational_devices_subquery = OperationalDevice.objects.filter(ip=OuterRef('ip'))
qs1 = (
AllowedDevice.objects
.all()
.annotate(
mac=Coalesce(
Subquery(operational_devices_subquery.values('mac')[:1]),
default_value,
output_field=null_char_field,
),
)
)
qs2 = (
OperationalDevice.objects
.exclude(
ip__in=qs1.values('ip'),
)
.annotate(
type=default_value,
owner=default_value,
)
)
final_qs = qs1.union(qs2)
多个领域的通用方法
更复杂但“通用”的方法可以使用Model._meta.get_fields()
. 对于“第二个”模型具有超过 1 个额外字段(不仅ip,mac
)的情况,它会更容易使用。示例代码(未经测试,但给人的印象一般):
# One more import:
from django.db.models.fields import NOT_PROVIDED
common_field_name = 'ip'
# OperationalDevice fields: ip, mac, some_more_fields ...
# AllowedDevice fields: ip, type, owner
operational_device_fields = OperationalDevice._meta.get_fields()
operational_device_fields_names = {_f.name for _f in operational_device_fields} # or set((_f.name for ...))
allowed_device_fields = AllowedDevice._meta.get_fields()
allowed_device_fields_names = {_f.name for _f in allowed_device_fields} # or set((_f.name for ...))
operational_devices_subquery = OperationalDevice.objects.filter(ip=OuterRef(common_field_name))
left_joined_qs = ( # "Kind-of". Assuming AllowedDevice to be "left" and OperationalDevice to be "right"
AllowedDevice.objects
.all()
.annotate(
**{
_f.name: Coalesce(
Subquery(operational_devices_subquery.values(_f.name)[1]),
Value(_f.get_default()), # Use defaults from model definition
output_field=_f,
)
for _f in operational_device_fields
if _f.name not in allowed_device_fields_names
# NOTE: if fields other than `ip` "overlap", then you might consider
# changing logic here. Current implementation keeps fields from the
# AllowedDevice
}
# Unpacked dict is partially equivalent to this:
# mac=Coalesce(
# Subquery(operational_devices_subquery.values('mac')[:1]),
# default_for_mac_eg_fallback_text_value,
# output_field=null_char_field,
# ),
# other_field = Coalesce(...),
# ...
)
)
lonely_right_rows_qs = (
OperationalDevice.objects
.exclude(
ip__in=AllowedDevice.objects.all().values(common_field_name),
)
.annotate(
**{
_f.name: Value(_f.get_default(), output_field=_f), # Use defaults from model definition
for _f in allowed_device_fields
if _f.name not in operational_device_fields_names
# NOTE: See previous NOTE
}
)
)
final_qs = left_joined_qs.union(lonely_right_rows_qs)
将 OneToOneField 用于“更好”的 SQL
理论上你可以使用device_info = models.OneToOneField(OperationalDevice, db_column='ip', primary_key=True, related_name='status_info')
: in AllowedDevice
。在这种情况下,您的第一个 QS 可以在不使用的情况下定义Subquery
:
from django.db.models import F
# Now 'ip' is not in field names ('device_info' is there), so add it:
allowed_device_fields_names.add(common_field_name)
# NOTE: I think this approach will result in a more compact SQL query without
# multiple `(SELECT "some_field" FROM device_info_table ... ) as "some-field"`.
# This also might result in better query performance.
honest_join_qs = (
AllowedDevice.objects
.all()
.annotate(
**{
_f.name: F(f'device_info__{_f.name}')
for _f in operational_device_fields
if _f.name not in allowed_device_fields_names
}
)
)
final_qs = honest_join_qs.union(lonely_right_rows_qs)
# or:
# final_qs = honest_join_qs.union(
# OperationalDevice.objects.filter(status_info__isnull=True).annotate(**missing_fields_annotation)
# )
# I'm not sure which approach is better performance-wise...
# Commented one will use something like:
# `SELECT ... FROM "device_info_table" LEFT OUTER JOIN "status_info_table" ON ("device_info_table"."ip" = "status_info_table"."ip") WHERE "status_info_table"."ip" IS NULL
#
# So it might be a little better than first with `union(QS.exclude(ip__in=honest_join_qs.values('ip'))`.
# Because later uses SQL like this:
# `SELECT ... FROM "device_info_table" WHERE NOT ip IN (SELECT ip FROM "status_info_table")`
#
# But it's better to measure timings of both approaches to be sure.
# @GrannyAching, can you compare them and tell in the comments which one is better ?
PS要自动化模型定义,您可以使用manage.py inspectdb
PPS 也许使用自定义的多表继承OneToOneField(..., parent_link=True)
可能比使用union
.
推荐阅读
- git - Git 分支过滤选择的文件
- oracle - ORA-02395: 超出了 IO 使用的调用限制并使用游标作为替代
- java - ChipNavigationBar如何初始化java代码
- ionic-framework - Ionic 5 离子载玻片作为“虚拟载玻片”
- fortran - 支持ieee_arithmetic的gfortran/ifort/nagfor/g95第一个版本是什么?
- python - 如何在 PySpark 中为大型数据集使用采样方法?
- node.js - 登录后没有出现任何错误的html文件
- c# - 使用命令行(在开发服务器中)添加对 .net 框架项目的包引用
- javascript - React/Gatsby 中的图表 js
- typescript - TypeScript 中没有浮动承诺检查