文档章节

Job Scheduling in a Distributed System Using Backfilling with Inaccurate Runtime Computations

猪迪
 猪迪
发布于 2017/08/14 00:49
字数 474
阅读 12
收藏 0

Two different scheduling techniques are considered and a simulation model
is used to evaluate system performance.

在本文中我们检查 a grid system where both parallel and sequential jobs require service.使用回填,but an error margin is added to a job’s runtime prediction.

队列网络模型:

系统中有三个到达流:

one at the GS (grid jobs 网格任务) and one inside each of the two sites (local jobs 本地任务).

一个gang可以有2~13个任务,一致分布。gang size=2,4,8,16

A job can start execution prior to a gang waiting in the queue if the following condition is met:

ServiceTime<=ElapsedTime+T

为实现回填方法,我们需要知道以下参数:

1)一个任务的服务时间

2)The exact time that all needed resources will be free for the gang to start execution.

SLD:平均slowdown

任务j的减速sj=响应时间rj/服务的时间ej

平均响应时间RT=sum(r_j)/m

wiki名词解释:Gang调度

Gang scheduling is used so that if two or more threads or processes communicate with each other, they will all be ready to communicate at the same time. If they were not gang-scheduled, then one could wait to send or receive a message to another while it is sleeping, and vice versa. When processors are over-subscribed(超额认购) and gang scheduling is not used within a group of processes or threads which communicate with each other, it can lead to situations where each communication event suffers the overhead of a context switch(上下文交换).

Gang scheduling is based on a data structure called the Ousterhout matrix(?). In this matrix each row represents a time slice(时间片), and each column a processor. The threads or processes of each job are packed into a single row of the matrix.[1] During execution, coordinated context switching is performed across all nodes to switch from the processes in one row to those in the next row.

Gang scheduling is stricter than coscheduling.[2] It requires all threads of the same process to run concurrently, while coscheduling allows for fragments, which are sets of threads that do not run concurrently with the rest of the gang.

Gang scheduling was implemented and used in production mode on several parallel machines, most notably the Connection Machine CM-5.

 

© 著作权归作者所有

共有 人打赏支持
猪迪
粉丝 6
博文 134
码字总数 180528
作品 0
海淀
程序员
私信 提问
机器学习框架:MXNet Python Overview Tutorial

Docs » MXNet Python Package » MXNet Python Overview Tutorial Edit on GitHub MXNet Python Overview Tutorial This page gives a general overview of MXNet’s python package. MXNet......

openthings
2016/01/10
389
0
KoaHub.JS用于Node.js的cron作业调度程序代码

node-schedule A cron-like and not-cron-like job scheduler for Node. Node Schedule Announcement: Node Schedule is looking for add additional collaborators with commit access. If ......

霁夜茶135
2016/10/28
171
0
Top Spark Ecosystem Projects(英)

Apache Spark has developed a rich ecosystem, including bothofficial and third party tools. We have a look at 5 third partyprojects which complement Spark in 5 different ways. By......

openthings
2016/03/10
13
0
实时计算框架 Spark: Lightning-fast cluster computing

real-time processing framework 实时计算框架 Real-time processing denotes processing, transforming and analyzing data on the fly Spark: Apache Spark has an advanced DAG executio......

swingcoder
2016/08/24
2
0
Celery: Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, ......

shaohan
2014/04/29
0
0

没有更多内容

加载失败,请刷新页面

加载更多

Elasticsearch节点角色类型node.master和node.data说明s

一般地,ElasticSearch集群中每个节点都有成为主节点的资格,也都存储数据,还可以提供查询服务。这些功能是由两个属性控制的(node.master和node.data)。默认情况下这两个属性的值都是tru...

傲娇字符
7分钟前
1
0
jdbcTemplate 分页 com.github.pagehelper.Page

由于项目中用了mybatis分页插件 com.github.pagehelper.Page 有时候复杂的查询分页想用jdbcTemplate。又想用分页。 分页对象 public class PageBean<T> implements Serializable { priv...

xiaomin0322
7分钟前
0
0
rem的使用

(function (doc, win) { var docEl = doc.documentElement var resizeEvt = 'orientationchange' in window ? 'orientationchange' : 'resize' var recalc = function () { var cl......

Js_Mei
14分钟前
0
0
idea 删除代码的注释

搜索栏使用 正则表达式搜索 (/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/|[ \t]*//.*) 会搜索出来所有注释的代码 用空格replace替换掉就可以了。 或者搜索 (/\*([^*]|[\r\n]|(\*+([^*/]|[\r\...

时刻在奔跑
17分钟前
2
0
eclipse maven 项目运行mvn clean 后无法运行

错误: 错误:找不到或无法加载主类com.yyy.test.Main 解决方法: “project” --"Clean" 参考链接:https://jingyan.baidu.com/article/cbcede07107d9802f40b4dff.html...

qimh
21分钟前
1
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部