时间戳

首页 > 解决方案 > 仅显示重复警告中的第一个 python

python - 仅显示重复警告中的第一个 python

问题描述

我正在做一个分布式进程，虽然我很欣赏这里和那里的一个很好的警告，但它往往会在每次工作人员接近它的内存限制时发送一个警告。这发生了很多。

只是想知道是否有一种聪明的方法可以不显示第一个警告并禁止所有重复的警告。鉴于该过程是分布式的，我假设不是，但值得一问。

from tsfresh.examples.robot_execution_failures import \
download_robot_execution_failures, \
load_robot_execution_failures
from tsfresh.feature_extraction import extract_features
from tsfresh.utilities.distribution import LocalDaskDistributor
import copy
import pandas as pd


download_robot_execution_failures()
df, y = load_robot_execution_failures()

df_comb = copy.deepcopy(df)
for x in range(100):
    df_iter = df
    df_iter['id'] = df_iter['id'] + df_comb['id'].max()
    df_comb = pd.concat([df_comb, df_iter], axis = 0)
    print(len(df_comb))

df_comb.reset_index(inplace = True, drop = True)


Distributor = LocalDaskDistributor(n_workers=3)

X = extract_features(timeseries_container=df_comb,
                     column_id='id', column_sort='time',
                     distributor=Distributor)

distributed.worker - WARNING - Memory use is high but worker has no data to store to disk.  Perhaps some other process is leaking memory?  Process memory: 94.87 GB -- Worker memory limit: 134.76 GB
distributed.worker - WARNING - Memory use is high but worker has no data to store to disk.  Perhaps some other process is leaking memory?  Process memory: 94.88 GB -- Worker memory limit: 134.76 GB
distributed.worker - WARNING - Memory use is high but worker has no data to store to disk.  Perhaps some other process is leaking memory?  Process memory: 95.02 GB -- Worker memory limit: 134.76 GB

标签： pythonpython-3.xsuppress-warnings

解决方案

这是你要找的吗？

from tsfresh.utilities.distribution import initialize_warnings_in_workers

initialize_warnings_in_workers(False)

这是这个方便的功能的作用：

def initialize_warnings_in_workers(show_warnings):
    """
    Small helper function to initialize warnings module in multiprocessing workers.

    On Windows, Python spawns fresh processes which do not inherit from warnings
    state, so warnings must be enabled/disabled before running computations.

    :param show_warnings: whether to show warnings or not.
    :type show_warnings: bool
    """
    warnings.catch_warnings()
    if not show_warnings:
        warnings.simplefilter("ignore")
    else:
        warnings.simplefilter("default")

推荐阅读