文档章节

about hint tracks

李有常
 李有常
发布于 2015/08/24 15:12
字数 994
阅读 226
收藏 0

This is a short description of RTP hint tracks that I created while I was working on L16 and Mpeg1/2 hint tracks.

Hint tracks are a series of instructions in a mp4 container file that tell a server how to transmit packets. Hint tracks always refer to another track, most likely an audio or video media track. 

This series of instructions tells the server when to send the packet, what type of RTP header to add, if there is any extra data in the packet, and how much data to send in the packet.  To save on space, a hint can contain a pointer to the media track, instead of duplicating that data.

It will also tell what SDP to add for the track that is referenced.

So, a file that is hinted should have a media track, and an associated hint track for that media track.  There are some mp4 container files that just have hint tracks - these are called "fat hints" and are usually not playable locally.  These are illegal in ISMA, so we won't talk about them.

To create hint tracks for a mp4 file is fairly simple with the mp4creator program.  Just execute the:

 mp4creator -hint=<track to hint track number> [-p <payload>] <mp4file>

For example:

[wmay@wmaytclinux2 content]$ mp4info xvid.mp4 
mp4info version 0.9.5.4
xvid.mp4:
Track    Type    Info
1    video    MPEG-4 Simple @ L3, 3.695 secs, 627 kbps, 608x256 @ 23.00 fps
2    od    Object Descriptors
3    scene    BIFS

I would execute the:

mp4creator -hint=1 xvid.mp4

and get:

[wmay@wmaytclinux2 content]$ mp4info xvid.mp4 
mp4info version 0.9.5.4
xvid.mp4:
Track    Type    Info
1    video    MPEG-4 Simple @ L3, 3.695 secs, 627 kbps, 608x256 @ 23.00 fps
2    od    Object Descriptors
3    scene    BIFS
4    hint    Payload MP4V-ES for track 1


-----------------------------------------------------------------------------

Now a bit about the internals of creating a hint track, for example, if you have your own codec and want to create a hint track for it.

The mp4v2 library has a complete set of functions that will allow you to create a hint track.

----

The first function is MP4AddHintTrack:

MP4TrackId MP4AddHintTrack(MP4FileHandle hFile, MP4TrackId refTrackId);

hFile is the file handle, refTrackId is the track ID that you want to add the hint for.  The return value is the hint track ID that you will use in the below functions.

----

The next step is to set the RTP Payload type for the track.  The following function does that:

bool MP4SetHintTrackRtpPayload(
    MP4FileHandle hFile,
    MP4TrackId hintTrackId,
    const char* pPayloadName,
    u_int8_t* pPayloadNumber,
    u_int16_t maxPayloadSize DEFAULT(0),
    const char *encode_params DEFAULT(NULL));

pPayloadName should be the payload type that will be shown in the a=rtpmap statment.

pPayloadNumber should most likely be MP4_SET_DYNAMIC_PAYLOAD (which will indicate a dynamic payload number), unless you have a payload defined in rfc 1890.  The return value (it is a pointer) will be the dynamic value.

maxPayloadSize will be the RTP payload size.

encode params are basically the channel number in the rtp map, if needed.

This will create the "m=" "a=rtpmap:/[/]" along with some other SDP for the media.

---

Now, we need to add a series of RtpHint and RtpPacket calls for each sample in the media.  Most functions will call MP4ReadSample for each of the samples in the reference track (MP4GetTrackNumberOfSamples).

MP4AddRtpHint or MP4AddRtpVideoHint is called each time the server should generate a new RTP Timestamp.

MP4AddRtpPacket is called for each RTP packet to be added.

When all information about the hint is called, MP4WriteRtpHint is called.

For example, if a video sample will span multiple packets, the calls might look like this:  

MP4AddRtpHint()
   MP4AddRtpPacket()
   ...
   MP4AddRtpPacket()
   ...
   MP4AddRtpPacket()
   ...
   MP4WriteRtpHint()

Or if you've got sample based audio (not frame based), the calls might look like:  

MP4AddRtpHint()
   MP4AddRtpPacket()
   MP4WriteRtpHint()
   MP4AddRtpHint()
   MP4AddRtpPacket()
   MP4WriteRtpHint()

Let's look at each call in detail:

bool MP4AddRtpHint( MP4FileHandle hFile, MP4TrackId hintTrackId);

is fairly simple.

bool MP4AddRtpPacket(MP4FileHandle hFile, MP4TrackId hintTrackId,
             bool setMbit DEFAULT(false),
                     int32_t transmitOffset DEFAULT(0));

is also pretty simple.  setMbit is to set the M bit in the RTP header.  transmitOffset is more of an advance topic - it's used to change the offset in the case of B frames - see the code in rfcisma.cpp for more information.

bool MP4WriteRtpHint(MP4FileHandle hFile, MP4TrackId hintTrackId,
             MP4Duration duration, bool isSyncSample DEFAULT(true));

where duration is the number of ticks in the timescale of the hint track involved in this RtpHint, and isSyncSample would be TRUE for sync samples, like I frames.

----

But what about data for the packets, you ask ?  Well there are a couple of functions for that. bool

MP4AddRtpImmediateData(MP4FileHandle hFile, MP4TrackId hintTrackId,
                    const u_int8_t* pBytes, u_int32_t numBytes);

can be used to add numBytes located at pBytes to the RTP packet.  This could be used for a codec specific RTP payload.

bool MP4AddRtpSampleData(MP4FileHandle hFile, MP4TrackId hintTrackId,
                 MP4SampleId sampleId, u_int32_t dataOffset,
                 u_int32_t dataLength);

is used to create a pointer to the reference track.  It creates a pointer to the sampleId given, at the dataOffset for dataLength bytes.

So, let's say we want to write a series of maxPayloadSize packets. We might do something like:   

MP4AddRtpHint();
    offset = 0;
   while (sampleSize > 0) {
       size_to_write = MIN(maxPayloadSize, sampleSize);
       // mbit is 0 except for last packet for timestamp
       MP4AddRtpPacket(file, hintTrack, (size_to_write == SampleSize) ? 1 : 0);
       MP4AddRtpSampleData(file, hintTrack, sampleId, offset, 
                           size_to_write);
       offset += size_to_write;
       sampleSize -= size_to_write;
   }
   MP4WriteRtpHint(file, hintTrack,, isSyncSample);

In the above, we may read into the sample to add immediate data.

Or if we want to pack several frames in a packet, we might do:

  sampleId = 1; // note 1 based sample id's
   offset = 0;
   while (sampleId <= MP4GetTrackNumberOfSamples(file, trackId)) {
      if (offset == 0) {
         MP4AddRtpHint();
         MP4AddRtpPacket();
      }
      size_to_write = MP4GetSampleSize(file, sampleId);
      if (size_to_write + offset < maxPayloadSize) {
         MP4AddSampleData(file, hintTrack, sampleId, offset, size_to_write);
         sampleId++;
         offset += size_to_write;
      } else {
MP4WriteRtpHint(file, hintTrack, duration of added tracks);
         offset = 0;
      }
   }

There are many variations on the above 2 examples.  See the files:

lib/mp4av/l16.cpp
          mpeg3.cpp
          rfcisma.cpp
          rfc2250.cpp
          rfc3119.cpp
          rfc3016.cpp

For more information and more ideas.


© 著作权归作者所有

李有常
粉丝 5
博文 126
码字总数 31866
作品 0
威海
后端工程师
私信 提问
基于JMF RTP的音视频传输

名流时尚服饰 dior 夏季 男装 男士t恤衬衫卫衣休闲裤牛仔裤英伦 socool 搜酷女包◆任选两款正价包包邮◆5周年店庆◆5折疯抢 紫紫 超人气包邮特价创意家居收纳压缩袋饰品服饰配件包包 socool...

嗯哼9925
2017/12/27
0
0
oracle database query Optimizer 查询优化

多表连接查询优化: LEADING Hint Description of the illustration leadinghint.gif (See "Specifying a Query Block in a Hint", tablespec::=) The hint instructs the optimizer to use ......

Oscarfff
2016/06/13
35
0
Oracle查询数据库的索引字段以及查询用索引

可以查询数据库的表上面的索引字段。 参考博文: http://www.dba-oracle.com/toracleindexhintsyntax.htm Question: I added an index hint in my query, but the hint is being ignored. W......

Oscarfff
2016/11/01
10
0
PgSQL · 特性分析 · Plan Hint

背景 有一个功能,是社区官方版”永远”不考虑引入的(参见PG TODO,查找”Oracle-style”),即类似Oracle的Plan Hint。社区开发者的理念是,引入Hint功能,会掩盖优化器本身的问题,导致缺陷...

阿里云RDS-数据库内核组
2016/01/10
0
0
IE8 提示 缺少')'

function domError(doml,hint=null){ var name = doml.attr('verify'); if(!hint){ hint = verify[name].hint; } doml.addClass('error'); doml.siblings('.from-hint').text(hint).show(); ......

小公牛的世界你们不懂
2017/01/21
281
3

没有更多内容

加载失败,请刷新页面

加载更多

Jenkins系列_插件安装及报错处理

进入Jenkins之后我们可以进行插件的安装,插件管理位于以下模块: 发现上面报了一堆错误,是因为插件的依赖没有安装好,那么这一节,就先把这些错误解决掉吧。解决完成后,也就基本会使用插件...

shzwork
今天
2
0
mysql mysql的所有查询语句和聚合函数(整理一下,忘记了可以随时看看)

查询所有字段 select * from 表名; 查询自定字段 select 字段名 from 表名; 查询指定数据 select * from 表名 where 条件; 带关键字IN的查询 select * from 表名 where 条件 [not] in(元素...

edison_kwok
昨天
9
0
解决多线程并行加载缓存问题(利用guava实现)

依赖 com.google.guava:guava:20.0 import com.google.common.cache.Cache;import com.google.common.cache.CacheBuilder;import java.util.concurrent.ExecutionException;import j......

暗中观察
昨天
3
0
利用VisualVM 内存查看

准备工作,建几个测试类。等下就是要查看这几个类里面的属性 package visualvm;public class MultiObject { private String str; private int i; MultiObject(String str...

冷基
昨天
2
0
组装一台工作游戏两用机

一、配置清单如下: 分类 项目 价格(元) 主板 华硕(ASUS)TUF Z370-PLUS GAMING II 电竞特工 Z370二代 支持9代CPU 1049 CPU 英特尔(Intel) i7 8700K 酷睿六核 盒装CPU处理器 2640 风扇 九...

mbzhong
昨天
4
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部