时间戳

首页 > 解决方案 > 计算python中元素组合的频率

python - 计算python中元素组合的频率

问题描述

我有以下df：

在此处输入图像描述

我想要做的是计算元素组合的频率。例如：

等等，换句话说，我需要生成这样的东西： 在此处输入图像描述

计算单个项目和组合项目的所有频率，并仅保留频率 >= n 的单个项目和组合项目，其中 n 是任何正整数。对于这个例子，假设 n -> {1, 2, 3, 4}。

我一直在尝试使用以下代码：

# candidates itemsets
records = []

# generates a list of lists of products that were bought together (convert df to list of lists)
for i in range(0, num_records):
    records.append([str(data.values[i,j]) for j in range(0, len(data.columns))])
    
# clean list (delete NaN values)
records = [[x for x in y if str(x) != 'nan'] for y in records]

OUTPUT:
[['detergent'],
 ['bread', 'water'],
 ['bread', 'umbrella', 'milk', 'diaper', 'beer'],
 ['detergent', 'beer', 'umbrella', 'milk'],
 ['cheese', 'detergent', 'diaper', 'umbrella'],
 ['umbrella', 'water', 'beer'],
 ['umbrella', 'water'],
 ['water', 'umbrella'],
 ['diaper', 'water', 'cheese', 'beer', 'detergent', 'umbrella'],
 ['umbrella', 'cheese', 'detergent', 'water', 'beer']]

接着：

setOfItems = []
newListOfItems = []
for item in records:
    if item in setOfItems:
        continue
    setOfItems.append(item)
    temp = list(item)
    occurence = records.count(item)
    temp.append(occurence)
    newListOfItems.append(temp)

OUTPUT:

['detergent', 1]
['bread', 'water', 1]
['bread', 'umbrella', 'milk', 'diaper', 'beer', 1]
['detergent', 'beer', 'umbrella', 'milk', 1]
['cheese', 'detergent', 'diaper', 'umbrella', 1]
['umbrella', 'water', 'beer', 1]
['umbrella', 'water', 1]
['water', 'umbrella', 1]
['diaper', 'water', 'cheese', 'beer', 'detergent', 'umbrella', 1]
['umbrella', 'cheese', 'detergent', 'water', 'beer', 1]

如您所见，它仅计算整行的频率（来自图像 1），但是我的预期输出是出现在第二个图像中的输出。

标签： pythonpandas

解决方案

有趣的问题！我itertools.combinations()用来生成所有可能的组合并collections.Counter()计算每个组合出现的频率：

import pandas as pd
import itertools
from collections import Counter

# create sample data
df = pd.DataFrame([
    ['detergent', np.nan],
    ['bread', 'water', None],
    ['bread', 'umbrella', 'milk', 'diaper', 'beer'],
    ['umbrella', 'water'],
    ['water', 'umbrella'],
    ['umbrella', 'water']
])

def get_all_combinations_without_nan_or_None(row):
    # remove nan, None and double values
    set_without_nan = {value for value in row if isinstance(value, str)}
    
    # generate all possible combinations of the values in a row
    all_combinations = []
    for i in range(1, len(set_without_nan)+1):
        result = list(itertools.combinations(set_without_nan, i))
        all_combinations.extend(result)
        
    return all_combinations
    
# get all posssible combinations of values in a row
all_rows = df.apply(get_all_combinations_without_nan_or_None, 1).values
all_rows_flatten = list(itertools.chain.from_iterable(all_rows))

# use Counter to count how many there are of each combination
count_combinations = Counter(all_rows_flatten)

文档collections.Counter()：
https ://docs.python.org/2/library/collections.html#collections.Counter

文档itertools.combinations()：
https ://docs.python.org/2/library/itertools.html#itertools.combinations

推荐阅读