python - Dask on single OSX machine - is it parallel by default?
问题描述
I have installed Dask on OSX Mojave. Does it execute computations in parallel by default? Or do I need to change some settings?
I am using the DataFrame API. Does that make a difference to the answer?
I installed it with pip. Does that make a difference to the answer?
解决方案
Yes, Dask is parallel by default.
Unless you specify otherwise, or create a distributed Client
, execution will happen with the "threaded" scheduler, in a number of threads equal to your number of cores. Note, however, that because of the python GIL (only one python instruction executed at a time), you may not get as much parallelism as available, depending on how good your specific tasks are at releasing the GIL. That is why you have a choice of schedulers.
Being on OSX, installing with pip: these make no difference. Using dataframes makes a difference in that it dictates the sorts of tasks you're likely running. Pandas is good at releasing the GIL for many operations.
推荐阅读
- c++ - 在虚幻引擎中删除 NXOpen 对象时崩溃
- html - 导航栏标题文本定位
- r - 使用多个数据集R的循环内的条件语句
- flutter - Flutter - 我如何将数据更新到 firebase 上的实时数据库并在我不知道的情况下找到这些键
- reactjs - React Router 中的 props.history.listen 需要在 useEffect 中清理吗?如果是这样,正确的方法是什么?
- azure-virtual-network - 点到站点 VPN 客户端是否应该支持对等 VNET?
- kivy - zipp 需要 Python '>=3.6' 但运行的 Python 是 2.7.17
- sql - 使用子查询的 in 语句问题
- sql - SAS SQL 使用带有宏变量的 where 条件
- typescript - 使用 webpack 运行 npm release 会导致错误