python - pandas 区域中最近匹配邻居的值总和
问题描述
我有两个数据框。
一个结构如下:
id,value
第二个:
id, neighbor_1, neighbor_2, neighbor_3, neighbor_4, neighbor_5, ...
现在我想id
在neighborhood
数据框中查找每个对应列的每个neighbor
,并计算一个 id 的所有邻居,以找到邻域中的聚合总和总值最大的 id。sum
value
total sum
import pandas as pd
from h3 import h3
k=1
df = pd.DataFrame({'x': {0: 16,
1: 17,
2: 18,
3: 19,
4: 20},
'y': {0: 48,
1: 49,
2: 50,
3: 51,
4: 52},
'value': {0: 2.0, 1: 4.0, 2: 100.0, 3: 40.0, 4: 500.0},
'id': {0: '891e15b706bffff',
1: '891e15b738fffff',
2: '891e15b714fffff',
3: '891e15b44c3ffff',
4: '891e15b448bffff'}})
display(df)
df_neighbors = df[['id']]
df_neighbors.index = df_neighbors['id']
df_neighbors = df_neighbors['id'].apply(lambda x: pd.Series(list(h3.k_ring(x,k))))
display(df_neighbors)
在 pandas 中计算此类问题(迭代连接和聚合)的有效方法是什么?
一个天真的解决方案:
import pandas as pd
from h3 import h3
import numpy as np
k=2
df = pd.DataFrame({'x': {0: 16,
1: 17,
2: 18,
3: 19,
4: 20},
'y': {0: 48,
1: 49,
2: 50,
3: 51,
4: 52},
'value': {0: 2.0, 1: 4.0, 2: 100.0, 3: 40.0, 4: 500.0},
'id': {0: '891e15b706bffff',
1: '891e15b738fffff',
2: '891e15b714fffff',
3: '891e15b44c3ffff',
4: '891e15b448bffff'}})
display(df)
df_neighbors = df[['id']]
df_neighbors.index = df_neighbors['id']
df_neighbors = df_neighbors['id'].apply(lambda x: pd.Series(list(h3.k_ring(x,k))))
display(df_neighbors)
joined = df.merge(df_neighbors.reset_index(), left_on='id', right_on='id', how='left')#.drop(['id_neighbors'], axis=1)
# display(joined)
for c in joined[df_neighbors.columns].columns:
joined[f'sum_of_{c}'] = joined.groupby([c]).value.transform(pd.Series.sum)
xx = [f'sum_of_{c}' for c in joined[df_neighbors.columns].columns]
joined['total_value_sum'] = joined[xx].sum(axis=1)
display(joined)
maximal_neighborhood = joined[df_neighbors.columns].iloc[joined.total_value_sum.argmax()]
display(maximal_neighborhood)
max_neighborhood_raw_elements = df[df['id'].isin(maximal_neighborhood)]
display(max_neighborhood_raw_elements)
avg_y_lat = np.average(max_neighborhood_raw_elements.y, weights=max_neighborhood_raw_elements.value)
avg_x_long = np.average(max_neighborhood_raw_elements.x, weights=max_neighborhood_raw_elements.value)
print(f'(x,y): ({avg_x_long},{avg_y_lat})')
解决方案
使用stack
I 可以在没有迭代的情况下进行操作,只有两个groupby
:
df_neighbors = df[['id']]
df_neighbors.index = df_neighbors['id']
df_neighbors = df_neighbors['id'].apply(lambda x: pd.Series(list(h3.k_ring(x,k)))).stack().to_frame('hexk').reset_index()#.reset_index(1, drop=True).reset_index()
#display(df_neighbors)
# df_neighbors.level_1.value_counts()
joined = df.merge(df_neighbors, left_on='id', right_on='id', how='left')#.drop(['id_neighbors'], axis=1)
#display(joined.head())
joined[f'sum_of_hexk'] = joined.groupby(['hexk']).value.transform(pd.Series.sum)
joined[f'total_value_sum'] = joined.groupby(['id']).sum_of_hexk.transform(pd.Series.sum)
#display(joined)
display(joined.total_value_sum.unique())
maximal_neighborhood = joined[joined.id == joined.iloc[joined.total_value_sum.argmax()].id].hexk
#display(maximal_neighborhood)
max_neighborhood_raw_elements = df[df['id'].isin(maximal_neighborhood)]
display(max_neighborhood_raw_elements)
avg_y_lat = np.average(max_neighborhood_raw_elements.y, weights=max_neighborhood_raw_elements.value)
avg_x_long = np.average(max_neighborhood_raw_elements.x, weights=max_neighborhood_raw_elements.value)
print(f'(x,y): ({avg_x_long},{avg_y_lat})')
推荐阅读
- c++ - 避免来自同一个项目的多个包含
- python - LSTM 模型需要很长时间
- android - 我正在使用 systrace 对其 GPU 的 android 应用程序进行性能调查。systrace 中的“GPU 完成”线程是什么意思
- dynamics-crm-2011 - 使用本地 Microsoft Dynamics CRM 进行客户端凭据身份验证的 OAuth 配置
- r - 根据某个变量将许多列转换为 NA
- excel - Excel vba selenium - 单击坐标
- angular - 使用 ionic cordova build android 时出错
- docker - 为什么多线程请求时请求响应延迟会增加?
- ajax - 搜索应用程序时出现 WSO2 API Manager Store 错误
- karate - 空手道 - 根据另一个静态数组从响应中过滤特定的 json 键