python - Python代码运行缓慢
问题描述
Python 代码运行速度极慢。开始很快,然后变成爬行。我能做些什么来加快速度吗?我正在提取一个文本文件,读取文件的内容,过滤文本文件的内容,然后将其写入 csv 供其他人用于 json。
我刚开始这样做,我相信你可以说出来。任何帮助将不胜感激。
import glob
from pathlib import Path
import datetime
import re
import csv
get_this = []
thislist = []
def timeteller():
now = datetime.datetime.now()
month = str('{:02d}'.format(now.month))
day1 = now.day -1
day = str('{:02d}'.format(day1))
year =str(now.year)
time =year+month+day
return time
def these_files(x, y):
configfiles = Path('O:/Unit Management/Reports/G4S/').glob('{}*/{}*Transaction.txt'.format(x, y))
for files in configfiles:
thislist.append(files)
return thislist
def hasNumbers(inputString):
numberfinal = []
numberfinal = re.findall("\d+", inputString)
if numberfinal == []:
numberfinal = '1'
return numberfinal
def get_odometers(thesepath):
for thispath in thesepath:
with open(thispath,"r") as f:
searchlines = f.readlines()
for i, line in enumerate(searchlines):
if "Odometers" in line:
get_this.append(line)
elif "Lifetime" in line:
get_this.append(line)
return get_this
def make_pretty(checkthis):
the_numbers = {}
the_numbers['Serial'] = banumber
for i, line in enumerate(checkthis):
the_numbers['Serial'] = banumber
if '(BNR) Odometers Accept' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['BNR'] = numberschecked[0]
elif '(BNR 2) Odometers Accept' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['BNR 2'] = numberschecked[0]
elif '(BCR) Odometers Accept' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['BCR'] = numberschecked[0]
elif '(BCR) Odometers Hopper1' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper1'] = numberschecked[0]
elif '(BCR) Odometers Hopper2' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper2'] = numberschecked[0]
elif '(BCR) Odometers Hopper3' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper3'] = numberschecked[0]
elif '(BCR) Odometers Hopper4' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper4'] = numberschecked[0]
elif '(BCR) Odometers Hopper5' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper5'] = numberschecked[0]
elif '(BCR) Odometers Hopper6' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper6'] = numberschecked[0]
elif '(BCR) Odometers Hopper7' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper7'] = numberschecked[0]
elif '(BCR) Odometers Hopper8' in line:
for l in checkthis[i:i+2]:
numbers = l[0:20]
numberschecked = hasNumbers(numbers)
the_numbers['Hopper8'] = numberschecked[0]
return the_numbers
intnow = int(timeteller())
intnow -= 1
now = str(intnow)
thelist = []
thispath = open('banumberlist1.txt')
finallist = []
for files in thispath:
getem = files
banumber = getem[0:8]
print(banumber)
combined = '{}_{}'.format(banumber,now)
thepaths = these_files(banumber, combined)
needtomakepretty = get_odometers(thepaths)
goeslast = make_pretty(needtomakepretty)
finallist.append(goeslast)
f = open ('odom01.txt', 'w')
for ba in finallist:
sba = str(ba)
f.write(sba)
解决方案
如果您想知道代码的哪一部分速度较慢,则需要使用分析器。
您可以使用cProfile
which 是标准库的一部分:
python -m cProfile -o profile.txt -s cumtime myscript.py
这会将分析结果写入profile.txt
,按累积时间对配置文件进行排序。这将使您大致了解您的程序在哪里花费时间。
要获得更详细的视图,您可以使用line_profiler模块。这甚至可以显示每行代码使用了多少时间。
安装后line_profiler
,将以下内容添加到脚本的开头:
import line_profiler
import atexit
profile = line_profiler.LineProfiler()
atexit.register(profile.print_stats)
现在将@profile
装饰器添加到所有函数中,如下所示:
@profile
def timeteller():
now = datetime.datetime.now()
month = str('{:02d}'.format(now.month))
day1 = now.day -1
day = str('{:02d}'.format(day1))
year =str(now.year)
time =year+month+day
return time
您应该将脚本末尾的内容放入一个函数(例如main
)并添加@profile
到该函数中。
现在运行您的脚本,您将看到一个配置文件。
推荐阅读
- python - 读取并合并具有不同列的多个 csv 文件
- sql - Aggregate expression issue with GROUPBY
- jquery - 从signalR动态加载的Jquery数据表
- c# - EF6 包括实体中集合的引用
- entity-framework - 扩展自动生成的招摇文档以包含特定的端点?
- python - 为什么用 Python 重写辛普森的积分规则会给出不同的结果?
- json - Golang 将 S3 中的 JSON 读入内存中的 struct
- ios - 计时器从时间戳开始计算时间
- flutter - 如何在颤动的列中展开容器而不溢出?
- r - 编写函数,计算列子集的行均值并创建列名