文档章节

Job Scheduling in a Distributed System Using Backfilling with Inaccurate Runtime Computations

猪迪
 猪迪
发布于 2017/08/14 00:49
字数 474
阅读 12
收藏 0

Two different scheduling techniques are considered and a simulation model
is used to evaluate system performance.

在本文中我们检查 a grid system where both parallel and sequential jobs require service.使用回填,but an error margin is added to a job’s runtime prediction.

队列网络模型:

系统中有三个到达流:

one at the GS (grid jobs 网格任务) and one inside each of the two sites (local jobs 本地任务).

一个gang可以有2~13个任务,一致分布。gang size=2,4,8,16

A job can start execution prior to a gang waiting in the queue if the following condition is met:

ServiceTime<=ElapsedTime+T

为实现回填方法,我们需要知道以下参数:

1)一个任务的服务时间

2)The exact time that all needed resources will be free for the gang to start execution.

SLD:平均slowdown

任务j的减速sj=响应时间rj/服务的时间ej

平均响应时间RT=sum(r_j)/m

wiki名词解释:Gang调度

Gang scheduling is used so that if two or more threads or processes communicate with each other, they will all be ready to communicate at the same time. If they were not gang-scheduled, then one could wait to send or receive a message to another while it is sleeping, and vice versa. When processors are over-subscribed(超额认购) and gang scheduling is not used within a group of processes or threads which communicate with each other, it can lead to situations where each communication event suffers the overhead of a context switch(上下文交换).

Gang scheduling is based on a data structure called the Ousterhout matrix(?). In this matrix each row represents a time slice(时间片), and each column a processor. The threads or processes of each job are packed into a single row of the matrix.[1] During execution, coordinated context switching is performed across all nodes to switch from the processes in one row to those in the next row.

Gang scheduling is stricter than coscheduling.[2] It requires all threads of the same process to run concurrently, while coscheduling allows for fragments, which are sets of threads that do not run concurrently with the rest of the gang.

Gang scheduling was implemented and used in production mode on several parallel machines, most notably the Connection Machine CM-5.

 

© 著作权归作者所有

共有 人打赏支持
猪迪
粉丝 6
博文 134
码字总数 180528
作品 0
海淀
程序员
机器学习框架:MXNet Python Overview Tutorial

Docs » MXNet Python Package » MXNet Python Overview Tutorial Edit on GitHub MXNet Python Overview Tutorial This page gives a general overview of MXNet’s python package. MXNet......

openthings
2016/01/10
389
0
Top Spark Ecosystem Projects(英)

Apache Spark has developed a rich ecosystem, including bothofficial and third party tools. We have a look at 5 third partyprojects which complement Spark in 5 different ways. By......

openthings
2016/03/10
13
0
KoaHub.JS用于Node.js的cron作业调度程序代码

node-schedule A cron-like and not-cron-like job scheduler for Node. Node Schedule Announcement: Node Schedule is looking for add additional collaborators with commit access. If ......

霁夜茶135
2016/10/28
171
0
实时计算框架 Spark: Lightning-fast cluster computing

real-time processing framework 实时计算框架 Real-time processing denotes processing, transforming and analyzing data on the fly Spark: Apache Spark has an advanced DAG executio......

swingcoder
2016/08/24
2
0
Celery: Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, ......

shaohan
2014/04/29
0
0

没有更多内容

加载失败,请刷新页面

加载更多

Spring中static变量不能@value注入的原因

今天本想使用@Value的方式使类中的变量获得yml文件中的配置值,然而一直失败,获得的一直为null。 类似于这样写的。 public class RedisShardedPool { private static ShardedJedisPool pool...

钟然千落
今天
2
0
CentOS7防火墙firewalld操作

firewalld Linux上新用的防火墙软件,跟iptables差不多的工具。 firewall-cmd 是 firewalld 的字符界面管理工具,firewalld是CentOS7的一大特性,最大的好处有两个:支持动态更新,不用重启服...

dingdayu
今天
1
0
关于组件化的最初步

一个工程可能会有多个版本,有国际版、国内版、还有针对各种不同的渠道化的打包版本、这个属于我们日常经常见到的打包差异化版本需求。 而对于工程的开发,比如以前的公司,分成了有三大块业...

DannyCoder
今天
2
0
Spring的Resttemplate发送带header的post请求

private HttpHeaders getJsonHeader() { HttpHeaders headers = new HttpHeaders(); MediaType type = MediaType.parseMediaType("application/json; charset=UTF-8"); ......

qiang123
昨天
3
0
Spring Cloud Gateway 之 Only one connection receive subscriber allowed

都说Spring Cloud Gateway好,我也来试试,可是配置了总是报下面这个错误: java.lang.IllegalStateException: Only one connection receive subscriber allowed. 困扰了我几天的问题,原来...

ThinkGem
昨天
40
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部