文档章节

Job Scheduling in a Distributed System Using Backfilling with Inaccurate Runtime Computations

猪迪
 猪迪
发布于 2017/08/14 00:49
字数 474
阅读 10
收藏 0
点赞 0
评论 0

Two different scheduling techniques are considered and a simulation model
is used to evaluate system performance.

在本文中我们检查 a grid system where both parallel and sequential jobs require service.使用回填,but an error margin is added to a job’s runtime prediction.

队列网络模型:

系统中有三个到达流:

one at the GS (grid jobs 网格任务) and one inside each of the two sites (local jobs 本地任务).

一个gang可以有2~13个任务,一致分布。gang size=2,4,8,16

A job can start execution prior to a gang waiting in the queue if the following condition is met:

ServiceTime<=ElapsedTime+T

为实现回填方法,我们需要知道以下参数:

1)一个任务的服务时间

2)The exact time that all needed resources will be free for the gang to start execution.

SLD:平均slowdown

任务j的减速sj=响应时间rj/服务的时间ej

平均响应时间RT=sum(r_j)/m

wiki名词解释:Gang调度

Gang scheduling is used so that if two or more threads or processes communicate with each other, they will all be ready to communicate at the same time. If they were not gang-scheduled, then one could wait to send or receive a message to another while it is sleeping, and vice versa. When processors are over-subscribed(超额认购) and gang scheduling is not used within a group of processes or threads which communicate with each other, it can lead to situations where each communication event suffers the overhead of a context switch(上下文交换).

Gang scheduling is based on a data structure called the Ousterhout matrix(?). In this matrix each row represents a time slice(时间片), and each column a processor. The threads or processes of each job are packed into a single row of the matrix.[1] During execution, coordinated context switching is performed across all nodes to switch from the processes in one row to those in the next row.

Gang scheduling is stricter than coscheduling.[2] It requires all threads of the same process to run concurrently, while coscheduling allows for fragments, which are sets of threads that do not run concurrently with the rest of the gang.

Gang scheduling was implemented and used in production mode on several parallel machines, most notably the Connection Machine CM-5.

 

© 著作权归作者所有

共有 人打赏支持
猪迪
粉丝 6
博文 50
码字总数 180528
作品 0
海淀
程序员
机器学习框架:MXNet Python Overview Tutorial

Docs » MXNet Python Package » MXNet Python Overview Tutorial Edit on GitHub MXNet Python Overview Tutorial This page gives a general overview of MXNet’s python package. MXNet......

openthings ⋅ 2016/01/10 ⋅ 0

Top Spark Ecosystem Projects(英)

Apache Spark has developed a rich ecosystem, including bothofficial and third party tools. We have a look at 5 third partyprojects which complement Spark in 5 different ways. By......

openthings ⋅ 2016/03/10 ⋅ 0

KoaHub.JS用于Node.js的cron作业调度程序代码

node-schedule A cron-like and not-cron-like job scheduler for Node. Node Schedule Announcement: Node Schedule is looking for add additional collaborators with commit access. If ......

霁夜茶135 ⋅ 2016/10/28 ⋅ 0

Celery: Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well. The execution units, ......

shaohan ⋅ 2014/04/29 ⋅ 0

实时计算框架 Spark: Lightning-fast cluster computing

real-time processing framework 实时计算框架 Real-time processing denotes processing, transforming and analyzing data on the fly Spark: Apache Spark has an advanced DAG executio......

swingcoder ⋅ 2016/08/24 ⋅ 0

.NET Open Source Developer Projects

This community maintained list showcases .NET open source developerprojects. It is intended to include projects that are useful for anyaspect of the development process. For con......

Yamazaki ⋅ 2015/08/26 ⋅ 0

100 open source Big Data architecture papers

Big Data technology has been extremely disruptive with open source playing a dominant role in shaping its evolution. While on one hand it has been disruptive, on the other it ha......

naughty ⋅ 2016/04/05 ⋅ 0

Spring+Quartz(一)

Quartz是一个非常优秀的任务调度引擎,详情请见官网:http://www.quartz-scheduler.org/ 而Spring很好地集成了Quartz,为企业级的任务调度提供了方便。 下面先看一个实现了Job接口的任务Hello...

YaZi-Liu ⋅ 2016/06/13 ⋅ 0

读取ftp上传过来的xls 之 ftp上传文件权限问题

[/RSS_DATA/omc172_0_0_110/deviceOffLineAndOnlineDuration/DeviceOffLineAndOnlineDuration_20180119.1.xls] is not existed 2018-01-22 22:32:45.004 ERROR [org.springframework.schedul......

yizhichao ⋅ 01/22 ⋅ 0

Spring定时任务的实现

近日项目开发中需要执行一些定时任务,比如需要在每天凌晨时候,分析一次前一天的日志信息,借此机会整理了一下定时任务的几种实现方式,由于项目采用spring框架,所以我都将结合 spring框架...

钟声已经敲响 ⋅ 2016/08/25 ⋅ 0

没有更多内容

加载失败,请刷新页面

加载更多

下一页

mysql5.7系列修改root默认密码

操作系统为centos7 64 1、修改 /etc/my.cnf,在 [mysqld] 小节下添加一行:skip-grant-tables=1 这一行配置让 mysqld 启动时不对密码进行验证 2、重启 mysqld 服务:systemctl restart mysql...

sskill ⋅ 6分钟前 ⋅ 0

Intellij IDEA神器常用技巧六-Debug详解

在调试代码的时候,你的项目得debug模式启动,也就是点那个绿色的甲虫启动服务器,然后,就可以在代码里面断点调试啦。下面不要在意,这个快捷键具体是啥,因为,这个keymap是可以自己配置的...

Mkeeper ⋅ 11分钟前 ⋅ 0

zip压缩工具、tar打包、打包并压缩

zip 支持压缩目录 1.在/tmp/目录下创建目录(study_zip)及文件 root@yolks1 study_zip]# !treetree 11└── 2 └── 3 └── test_zip.txt2 directories, 1 file 2.yum...

蛋黄Yolks ⋅ 13分钟前 ⋅ 0

聊聊HystrixThreadPool

序 本文主要研究一下HystrixThreadPool HystrixThreadPool hystrix-core-1.5.12-sources.jar!/com/netflix/hystrix/HystrixThreadPool.java /** * ThreadPool used to executed {@link Hys......

go4it ⋅ 32分钟前 ⋅ 0

容器之上传镜像到Docker hub

Docker hub在国内可以访问,首先要创建一个账号,这个后面会用到,我是用126邮箱注册的。 1. docker login List-1 Username不能使用你注册的邮箱,要用使用注册时用的username;要输入密码 ...

汉斯-冯-拉特 ⋅ 38分钟前 ⋅ 0

SpringBoot简单使用ehcache

1,SpringBoot版本 2.0.3.RELEASE ①,pom.xml <parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.0.3.RELE......

暗中观察 ⋅ 39分钟前 ⋅ 0

监控各项服务

比如有三个服务, 为了减少故障时间,增加监控任务,使用linux的 crontab 实现. 步骤: 1,每个服务写一个ping接口 监控如下内容: 1,HouseServer 是否正常运行,所以需要增加一个ping的接口 ; http...

黄威 ⋅ 42分钟前 ⋅ 0

Spring源码解析(八)——实例创建(下)

前言 来到实例创建的最后一节,前面已经将一个实例通过不同方式(工厂方法、构造器注入、默认构造器)给创建出来了,下面我们要对创建出来的实例进行一些“加工”处理。 源码解读 回顾下之前...

MarvelCode ⋅ 43分钟前 ⋅ 0

nodejs __proto__跟prototype

前言 nodejs中完全没有class的这个概念,这点跟PHP,JAVA等面向对象的语言很不一样,没有class跟object的区分,那么nodejs是怎么样实现继承的呢? 对象 对象是由属性跟方法组成的一个东西,就...

Ai5tbb ⋅ 49分钟前 ⋅ 0

Ubuntu16.04 PHP7.0 不能用MYSQLi方式连接MySQL5.7数据库

Q: Ubuntu16.04 PHP7.0 不能用MYSQLi方式连接MySQL5.7数据库 A: 执行以下2条命令解决: apt-get install php-mysql service apache2 restart php -m 执行后会多以下4个模块: mysqli mysqlnd...

SamXIAO ⋅ 51分钟前 ⋅ 0

没有更多内容

加载失败,请刷新页面

加载更多

下一页

返回顶部
顶部