文档章节

Job Scheduling in a Distributed System Using Backfilling with Inaccurate Runtime Computations

猪迪
 猪迪
发布于 2017/08/14 00:49
字数 474
阅读 10
收藏 0

Two different scheduling techniques are considered and a simulation model
is used to evaluate system performance.

在本文中我们检查 a grid system where both parallel and sequential jobs require service.使用回填,but an error margin is added to a job’s runtime prediction.

队列网络模型:

系统中有三个到达流:

one at the GS (grid jobs 网格任务) and one inside each of the two sites (local jobs 本地任务).

一个gang可以有2~13个任务,一致分布。gang size=2,4,8,16

A job can start execution prior to a gang waiting in the queue if the following condition is met:

ServiceTime<=ElapsedTime+T

为实现回填方法,我们需要知道以下参数:

1)一个任务的服务时间

2)The exact time that all needed resources will be free for the gang to start execution.

SLD:平均slowdown

任务j的减速sj=响应时间rj/服务的时间ej

平均响应时间RT=sum(r_j)/m

wiki名词解释:Gang调度

Gang scheduling is used so that if two or more threads or processes communicate with each other, they will all be ready to communicate at the same time. If they were not gang-scheduled, then one could wait to send or receive a message to another while it is sleeping, and vice versa. When processors are over-subscribed(超额认购) and gang scheduling is not used within a group of processes or threads which communicate with each other, it can lead to situations where each communication event suffers the overhead of a context switch(上下文交换).

Gang scheduling is based on a data structure called the Ousterhout matrix(?). In this matrix each row represents a time slice(时间片), and each column a processor. The threads or processes of each job are packed into a single row of the matrix.[1] During execution, coordinated context switching is performed across all nodes to switch from the processes in one row to those in the next row.

Gang scheduling is stricter than coscheduling.[2] It requires all threads of the same process to run concurrently, while coscheduling allows for fragments, which are sets of threads that do not run concurrently with the rest of the gang.

Gang scheduling was implemented and used in production mode on several parallel machines, most notably the Connection Machine CM-5.

 

© 著作权归作者所有

共有 人打赏支持
猪迪
粉丝 6
博文 134
码字总数 180528
作品 0
海淀
程序员
机器学习框架:MXNet Python Overview Tutorial

Docs » MXNet Python Package » MXNet Python Overview Tutorial Edit on GitHub MXNet Python Overview Tutorial This page gives a general overview of MXNet’s python package. MXNet......

openthings
2016/01/10
389
0
KoaHub.JS用于Node.js的cron作业调度程序代码

node-schedule A cron-like and not-cron-like job scheduler for Node. Node Schedule Announcement: Node Schedule is looking for add additional collaborators with commit access. If ......

霁夜茶135
2016/10/28
171
0
Top Spark Ecosystem Projects(英)

Apache Spark has developed a rich ecosystem, including bothofficial and third party tools. We have a look at 5 third partyprojects which complement Spark in 5 different ways. By......

openthings
2016/03/10
13
0
Celery: Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, ......

shaohan
2014/04/29
0
0
实时计算框架 Spark: Lightning-fast cluster computing

real-time processing framework 实时计算框架 Real-time processing denotes processing, transforming and analyzing data on the fly Spark: Apache Spark has an advanced DAG executio......

swingcoder
2016/08/24
2
0

没有更多内容

加载失败,请刷新页面

加载更多

下一页

docker中安装了RabbitMQ后无法访问其Web管理页面

在官网找了"$ docker run -d --hostname my-rabbit --name some-rabbit -p 8080:15672 rabbitmq:3-management"这条安装命令,在docker上安装了RabbitMQ,,结果输入http://localhost:8080并不......

钟然千落
25分钟前
0
0
spring-cloud | 分布式session共享

写在前面的话 各位小伙伴,你们有福了,这一节不仅教大家怎么实现分布式session的问题,还用kotlin开发,喜欢kotlin的小伙伴是不是很开心! 以前在写Android的时候,就对客户端请求有一定的认...

冯文议
44分钟前
0
0
c语言之内存分配笔记

先看一个数组: short array[5] = {1,2} // 这儿定义的一个int类型的数组,数组第1和第2个元素值是1和2.其余后面默认会给值为0; 或者 short array[] = {1,2};//这儿数组第1和第2个元素,数组...

DannyCoder
今天
4
0
Shell | linux安装包不用选择Y/N的方法

apt-get install -y packageOR echo "y" | sudo apt-get install package

云迹
今天
2
0
Hadoop的大数据生态圈

基于Hadoop的大数据的产品圈 大数据产品的一句话概括 Apache Hadoop: 是Apache开源组织的一个分布式计算开源框架,提供了一个分布式文件系统子项目(HDFS)和支持MapReduce分布式计算的软件架...

zimingforever
今天
7
0

没有更多内容

加载失败,请刷新页面

加载更多

下一页

返回顶部
顶部