现在来分析live555中关于H264的处理部分,主要包括从文件中读取数据进行并进行frame(NALU)的分割,然后对frame进行分片,这些工作都是在frame交给RTP sink之前完成的。
[cpp]?
view plain?
copy
?
print
?
- void?MultiFramedRTPSink::packFrame()?{??
- ??if?(fOutBuf->haveOverflowData())?{??
- ...??
- ??}?else?{??
- ??
- ??
- ????//??
- ????//从source中获取下一个frame??
- ????fSource->getNextFrame(fOutBuf->curPtr(),?fOutBuf->totalBytesAvailable(),??
- ??????????????afterGettingFrame,?this,?ourHandleClosure,?this);??
- ??}??
- } ?
?? ?getNextFrame是定义在FramedSource中的非虚函数,从source中获取下一个frame,然后调用回调函数afterGettingFrame。afterGettingFrame被定义为静态函数,因为在C++中类成员函数是不能作为回调用函数的。不过这里为什么要用回调函数回?
?? ?注意,对于H264来说,上面的fSource并不是MPEGVideoStreamFramer类,因为在 H264VideoRTPSink::continuePlaying()函数中改变了fSource的值。 ? ?
[cpp]?
view plain
?copy
?print
?
- Boolean?H264VideoRTPSink::continuePlaying()?{??
- ??//?First,?check?whether?we?have?a?'fragmenter'?class?set?up?yet.??
- ??//?If?not,?create?it?now:??
- ??if?(fOurFragmenter?==?NULL)?{??
- ????//创建一个辅助类H264FUAFragmenter,用于H264按照RFC3984进行RTP打包??
- ????fOurFragmenter?=?new?H264FUAFragmenter(envir(),?fSource,?OutPacketBuffer::maxSize,??
- ???????????????????????ourMaxPacketSize()?-?12/*RTP?hdr?size*/);??
- ????fSource?=?fOurFragmenter;??
- ??}??
- ??//?Then?call?the?parent?class's?implementation:??
- ??return?MultiFramedRTPSink::continuePlaying();??
- } ?
?? ?fSource被指向了H264FUAFragmenter类,这个类主要实现了H264按照RFC3984进行RTP分包,不过这里的实现每个RTP中最多只包含一个NALU,没有实现组合封包的情形。这个类的继承关系如下:H264FUAFragmenter->FramedFilter->FramedSource。很明显,这是一个filter,包装了MPEGVideoStreamFramer类的对像。
?? ?先来看来看getNextFrame的实现
[cpp]?
view plain
?copy
?print
?
- void?FramedSource::getNextFrame(unsigned?char*?to,?unsigned?maxSize,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ????????????????afterGettingFunc*?afterGettingFunc,??
- ????????????????void*?afterGettingClientData,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ????????????????onCloseFunc*?onCloseFunc,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ????????????????void*?onCloseClientData)?{??
- ??//?Make?sure?we're?not?already?being?read:??
- ??if?(fIsCurrentlyAwaitingData)?{??
- ????envir()?<<?"FramedSource["?<<?this?<<?"]::getNextFrame():?attempting?to?read?more?than?once?at?the?same?time!\n";??
- ????envir().internalError();??
- ??fTo?=?to;?????????????//buffer地址??
- ??fMaxSize?=?maxSize;???//buffer最大长度??
- ??fNumTruncatedBytes?=?0;?//?by?default;?could?be?changed?by?doGetNextFrame()??
- ??fDurationInMicroseconds?=?0;?//?by?default;?could?be?changed?by?doGetNextFrame()??
- ??fAfterGettingFunc?=?afterGettingFunc;?????????????//获取完一个frame后将执行这个函数??
- ??fAfterGettingClientData?=?afterGettingClientData;?//这个参数就是MultiFramedRTPSink类型指针??
- ??fOnCloseFunc?=?onCloseFunc;??
- ??fOnCloseClientData?=?onCloseClientData;??
- ??fIsCurrentlyAwaitingData?=?True;??
- ??doGetNextFrame();??
- } ?
?? ?上面的函数主要是进行一些成员变量的初始化,获取到的frame需要保存到fTo地址中,然后调用fAfterGettingFunc函数,若文件读取完毕,还需要调用fOnCloseFunc函数。重要的工作还是在doGetNextFrame函数中完成,不过它是定义在FramedSource类中的纯虚函数,需要在子类中重新实现。
?? ?现在来看H264FUAFragmenter中对doGetNextFrame的实现
[cpp]?
view plain
?copy
?print
?
- void?H264FUAFragmenter::doGetNextFrame()?{??
- ??if?(fNumValidDataBytes?==?1)?{??
- ??????//读取一个新的frame??
- ????//?We?have?no?NAL?unit?data?currently?in?the?buffer.??Read?a?new?one:??
- ????fInputSource->getNextFrame(&fInputBuffer[1],?fInputBufferSize?-?1,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????afterGettingFrame,??
- ???????????????????FramedSource::handleClosure,?this);??
- ??????//??
- ??????//现在buffer中已经存在NALU数据了,需要考虑三种情况??
- ??????//1.一个新的NALU,且足够小能投递给RTP?sink。??
- ??????//2.一个新的NALU,但是比RTP?sink要求的包大了,投递第一个分片作为一个FU-A?packet,?并带上一个额外的头字节。??
- ??????//3.部分NALU数据,投递下一个分片作为一个FU-A?packet,并带上2个额外的头字节。??
- ????//?We?have?NAL?unit?data?in?the?buffer.??There?are?three?cases?to?consider:??
- ????//?1.?There?is?a?new?NAL?unit?in?the?buffer,?and?it's?small?enough?to?deliver??
- ????//????to?the?RTP?sink?(as?is).??
- ????//?2.?There?is?a?new?NAL?unit?in?the?buffer,?but?it's?too?large?to?deliver?to??
- ????//????the?RTP?sink?in?its?entirety.??Deliver?the?first?fragment?of?this?data,??
- ????//????as?a?FU-A?packet,?with?one?extra?preceding?header?byte.??
- ????//?3.?There?is?a?NAL?unit?in?the?buffer,?and?we've?already?delivered?some??
- ????//????fragment(s)?of?this.??Deliver?the?next?fragment?of?this?data,??
- ????if?(fMaxSize?<?fMaxOutputPacketSize)?{?//?shouldn't?happen??
- ??????envir()?<<?"H264FUAFragmenter::doGetNextFrame():?fMaxSize?("??
- ??????????<<?fMaxSize?<<?")?is?smaller?than?expected\n";??
- ????}?else?{??
- ??????fMaxSize?=?fMaxOutputPacketSize;??
- ????}??
- ????fLastFragmentCompletedNALUnit?=?True;?//?by?default??
- ????if?(fCurDataOffset?==?1)?{?//?case?1?or?2??
- ??????if?(fNumValidDataBytes?-?1?<=?fMaxSize)?{?//?case?1??
- ??????????//??
- ??????????//情况1,?处理整个NALU??
- ????memmove(fTo,?&fInputBuffer[1],?fNumValidDataBytes?-?1);??
- ????fFrameSize?=?fNumValidDataBytes?-?1;??
- ????fCurDataOffset?=?fNumValidDataBytes;??
- ??????}?else?{?//?case?2??
- ??????????//??
- ??????????//情况2,处理NALU的第1个分片。注意,我们添加FU指示符和FU头字节(with?S?bit)到包的最前面(??
- ??????????//重用已经存在的NAL?头字节作为FU的头字节)??
- ????//?We?need?to?send?the?NAL?unit?data?as?FU-A?packets.??Deliver?the?first??
- ????//?packet?now.??Note?that?we?add?FU?indicator?and?FU?header?bytes?to?the?front??
- ????//?of?the?packet?(reusing?the?existing?NAL?header?byte?for?the?FU?header).??
- ????fInputBuffer[0]?=?(fInputBuffer[1]?&?0xE0)?|?28;?//?FU?indicator??
- ????fInputBuffer[1]?=?0x80?|?(fInputBuffer[1]?&?0x1F);?//?FU?header?(with?S?bit)???重用NALU头字节??
- ????fFrameSize?=?fMaxSize;??
- ????fCurDataOffset?+=?fMaxSize?-?1;??
- ????fLastFragmentCompletedNALUnit?=?False;??
- ??????}??
- ????}?else?{?//?case?3??
- ????????//??
- ????????//情况3,处理非第1个分片。需要添加FU指示符和FU头(我们重用了第一个分片中的字节,但是需要清除S位,??
- ????????//并在最后一个分片中添加E位)??
- ????????//??
- ??????//?We?are?sending?this?NAL?unit?data?as?FU-A?packets.??We've?already?sent?the??
- ??????//?first?packet?(fragment).??Now,?send?the?next?fragment.??Note?that?we?add??
- ??????//?FU?indicator?and?FU?header?bytes?to?the?front.??(We?reuse?these?bytes?that??
- ??????//?we?already?sent?for?the?first?fragment,?but?clear?the?S?bit,?and?add?the?E??
- ??????//?bit?if?this?is?the?last?fragment.)??
- ??????fInputBuffer[fCurDataOffset-2]?=?fInputBuffer[0];?//?FU?indicator??
- ??????fInputBuffer[fCurDataOffset-1]?=?fInputBuffer[1]&~0x80;?//?FU?header?(no?S?bit)??
- ??????unsigned?numBytesToSend?=?2?+?fNumValidDataBytes?-?fCurDataOffset;??
- ??????if?(numBytesToSend?>?fMaxSize)?{??
- ????//?We?can't?send?all?of?the?remaining?data?this?time:??
- ????numBytesToSend?=?fMaxSize;??
- ????fLastFragmentCompletedNALUnit?=?False;??
- ??????}?else?{??
- ????//最后一个分片,需要在FU头中设置E标志位??
- ????//?This?is?the?last?fragment:??
- ????fInputBuffer[fCurDataOffset-1]?|=?0x40;?//?set?the?E?bit?in?the?FU?header??
- ????fNumTruncatedBytes?=?fSaveNumTruncatedBytes;??
- ??????}??
- ??????memmove(fTo,?&fInputBuffer[fCurDataOffset-2],?numBytesToSend);??
- ??????fFrameSize?=?numBytesToSend;??
- ??????fCurDataOffset?+=?numBytesToSend?-?2;??
- ????}??
- ????if?(fCurDataOffset?>=?fNumValidDataBytes)?{??
- ??????//?We're?done?with?this?data.??Reset?the?pointers?for?receiving?new?data:??
- ??????fNumValidDataBytes?=?fCurDataOffset?=?1;??
- ????//?Complete?delivery?to?the?client:??
- ????FramedSource::afterGetting(this);??
- } ?
?? ?H264FUAFragmenter::doGetNextFrame函数第一次执行时,执行条件1,需要调用 MPEGVideoStreamFramer::doGetNextFrame读取一个新的frame,获取frame的具体过程稍后再分析。现在先看获取frame之后的工作,afterGettingFrame函数
[cpp]?
view plain
?copy
?print
?
- void?H264FUAFragmenter::afterGettingFrame(void*?clientData,?unsigned?frameSize,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ??????????????????????unsigned?numTruncatedBytes,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ??????????????????????struct?timeval?presentationTime,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ??????????????????????unsigned?durationInMicroseconds)?{??
- ??H264FUAFragmenter*?fragmenter?=?(H264FUAFragmenter*)clientData;??
- ??fragmenter->afterGettingFrame1(frameSize,?numTruncatedBytes,?presentationTime,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ?????????????????durationInMicroseconds);??
- } ?
没什么好说的,再看afterGettingFrame1函数
[cpp]?
view plain
?copy
?print
?
- void?H264FUAFragmenter::afterGettingFrame1(unsigned?frameSize,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????????unsigned?numTruncatedBytes,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????????struct?timeval?presentationTime,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????????unsigned?durationInMicroseconds)?{??
- ??fNumValidDataBytes?+=?frameSize;??????//保存读到的frame长度??
- ??fSaveNumTruncatedBytes?=?numTruncatedBytes;??
- ??fPresentationTime?=?presentationTime;??
- ??fDurationInMicroseconds?=?durationInMicroseconds;??
- ??//?Deliver?data?to?the?client:??
- } ?
?? ?上面的代码首先记录几个数据到成员变量中,fNumValidDataBytes很重要,表示读取到的frame长度+1。然后又一次调用了H264FUAFragmenter::doGetNextFrame(),这里将进入H264FUAFragmenter::doGetNextFrame函数中第二个条件分支,这种循环调用很容易把人弄迷糊了。
?? ?H264FUAFragmenter::doGetNextFrame函数中第二个条件分支中,处理H264的RTP分片问题,这里是按照RFC3984进行RTP封装的。你应该注意到,在上篇文章"RTP的打包与发送"中,也出现了分片的代码(MultiFramedRTPSink::packFrame函数中),那里直接将frame按MTU的长度来拆分。那为什么H264还要自定义一套RTP打包的标准呢?暂时我也不清楚。
?? ?在H264FUAFragmenter::doGetNextFrame()最后调用了 FramedSource::afterGetting
[cpp]?
view plain
?copy
?print
?
- void?FramedSource::afterGetting(FramedSource*?source)?{??
- ??source->fIsCurrentlyAwaitingData?=?False;?????//表示已经获取到数据了,处于非等待状态??
- ??????//?indicates?that?we?can?be?read?again??
- ??????//?Note?that?this?needs?to?be?done?here,?in?case?the?"fAfterFunc"??
- ??????//?called?below?tries?to?read?another?frame?(which?it?usually?will)??
- ??//通过回调用进行后续处理??
- ??if?(source->fAfterGettingFunc?!=?NULL)?{??
- ????(*(source->fAfterGettingFunc))(source->fAfterGettingClientData,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????source->fFrameSize,?source->fNumTruncatedBytes,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????source->fPresentationTime,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ???????????????????source->fDurationInMicroseconds);??
- } ?
?? ?上面的代码主要是调用了FramedSource::getNextFrame函数中传递下来的回调函数,这个回调函数就是MultiFramedRTPSink::afterGettingFrame,处理过程在上一篇文章"RTP的打包与发送"中已经分析过了。
?? ?现在来看MPEGVideoStreamFramer::doGetNextFrame获取Frame的过程。继承关系:H264VideoStreamFramer->MPEGVideoStreamFramer->FramedFilter->FramedSource。在继承路径中存在FrameFilter,这说明H264VideoStreamFramer包装了其它source(包装的是读取文件的字节流source)。doGetNextFrame函数首先在MPEGVideoStreamFramer中实现。
[cpp]?
view plain
?copy
?print
?
- void?MPEGVideoStreamFramer::doGetNextFrame()?{??
- ??fParser->registerReadInterest(fTo,?fMaxSize);?//将目的buffer信息注册到语法分析类中??
- ??continueReadProcessing();?????//继续进行读数据??
- } ?
?? ?这里的MPEGVideoStreamFramer::fParser,是一个MPEGVideoStreamParser类型指针,作为语法分析器。再来看continueReadProcessing函数
[cpp]?
view plain
?copy
?print
?
- void?MPEGVideoStreamFramer::continueReadProcessing()?{??
- ??unsigned?acquiredFrameSize?=?fParser->parse();????//文件的语法分析(即demux)??
- ??if?(acquiredFrameSize?>?0)?{??
- ????//?We?were?able?to?acquire?a?frame?from?the?input.??
- ????//?It?has?already?been?copied?to?the?reader's?space.??
- ????fFrameSize?=?acquiredFrameSize;??
- ????fNumTruncatedBytes?=?fParser->numTruncatedBytes();??
- ????//?"fPresentationTime"?should?have?already?been?computed.??
- ????//?Compute?"fDurationInMicroseconds"?now:??
- ????fDurationInMicroseconds??
- ??????=?(fFrameRate?==?0.0?||?((int)fPictureCount)?<?0)???0??
- ??????:?(unsigned)((fPictureCount*1000000)/fFrameRate);??
- #ifdef?DEBUG??
- ????fprintf(stderr,?"%d?bytes?@%u.%06d,?fDurationInMicroseconds:?%d?((%d*1000000)/%f)\n",?acquiredFrameSize,?fPresentationTime.tv_sec,?fPresentationTime.tv_usec,?fDurationInMicroseconds,?fPictureCount,?fFrameRate);??
- #endif??
- ????fPictureCount?=?0;??
- ??????
- ????//调用自身的afterGetting函数,因为这不一个"leaf"?source,?所以可能直接调用,??
- ????//而不用担心出现无限递归??
- ????//??
- ????//?Call?our?own?'after?getting'?function.??Because?we're?not?a?'leaf'??
- ????//?source,?we?can?call?this?directly,?without?risking?infinite?recursion.??
- ????afterGetting(this);??
- ??}?else?{??
- ????//?We?were?unable?to?parse?a?complete?frame?from?the?input,?because:??
- ????//?-?we?had?to?read?more?data?from?the?source?stream,?or??
- ????//?-?the?source?stream?has?ended.??
- } ?
?? ?函数中首先调用了MPEGVideoStreamParser::parse函数,将一个完整的Frame分析出来,并copy到了fTo(fTo就是OutPacketBuffer中的缓冲区)中,这其中肯定也实现了从文件中读取数据的过程。这里的fNumTruncatedBytes变量需要注意,fNumTruncatedBytes>0的话,表明Frame的实际长度大于fTo的最大长度,这将导致数据丢失,这时就要考虑增加缓冲区的长度了。成功获取一个Frame后,将调用afterGetting函数处理后续工作。
?? ?先来看parse函数,parse是定义在MPEGVideoStreamParser中的纯虚函数,在子类H264VideoStreamParser中实现。parse主要是从文件的字节流中,分离出一个个的Frame,对于H264而言其实就是对一个个的NALU。*.264文件的格式非常简单,每个NALU以 0x00000001 作为起始符号(中间的NALU也可以以0x000001作为起始符),顺序存放。
[cpp]?
view plain
?copy
?print
?
- unsigned?H264VideoStreamParser::parse()?{??
- ??try?{??
- ????//首先找到起始符号,?并跳过。文件流的最开始必需以0x00000001开始,但后续的NALU充许以0x000001(3?bytes)作为分隔??
- ????//?The?stream?must?start?with?a?0x00000001:??
- ????if?(!fHaveSeenFirstStartCode)?{??
- ??????//?Skip?over?any?input?bytes?that?precede?the?first?0x00000001:??
- ??????u_int32_t?first4Bytes;??
- ??????while?((first4Bytes?=?test4Bytes())?!=?0x00000001)?{??
- ????get1Byte();?setParseState();?//?ensures?that?we?progress?over?bad?data??
- ??????skipBytes(4);?//?skip?this?initial?code??
- ????????
- ??????setParseState();??
- ??????fHaveSeenFirstStartCode?=?True;?//?from?now?on??
- ????//将起始标志也保存到目的缓冲区中??
- ????if?(fOutputStartCodeSize?>?0)?{??
- ??????//?Include?a?start?code?in?the?output:??
- ??????save4Bytes(0x00000001);?????
- ????//保存所有数据,直至遇到起始标志,或者文件结束符。需要注意NALU中的第一个字节,因为它指示了NALU的类型??
- ????//?Then?save?everything?up?until?the?next?0x00000001?(4?bytes)?or?0x000001?(3?bytes),?or?we?hit?EOF.??
- ????//?Also?make?note?of?the?first?byte,?because?it?contains?the?"nal_unit_type":???
- ????u_int8_t?firstByte;??
- ????if?(haveSeenEOF())?{??
- ???????//??
- ???????//已经设置了文件结束标志,将剩下的数据保存也来即可??
- ??????//?We?hit?EOF?the?last?time?that?we?tried?to?parse?this?data,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ??????//?so?we?know?that?the?remaining?unparsed?data?forms?a?complete?NAL?unit:??
- ??????unsigned?remainingDataSize?=?totNumValidBytes()?-?curOffset();??
- ??????if?(remainingDataSize?==?0)?(void)get1Byte();?//?forces?another?read,?which?will?cause?EOF?to?get?handled?for?real?this?time??
- #ifdef?DEBUG??
- ??????fprintf(stderr,?"This?NAL?unit?(%d?bytes)?ends?with?EOF\n",?remainingDataSize);??
- #endif??
- ??????if?(remainingDataSize?==?0)?return?0;??
- ??????firstByte?=?get1Byte();???//将第一个字节保存下来,其指示了NALU的类型??
- ??????saveByte(firstByte);??
- ??????while?(--remainingDataSize?>?0)?{??
- ????saveByte(get1Byte());??
- ??????u_int32_t?next4Bytes?=?test4Bytes();??
- ??????firstByte?=?next4Bytes>>24;???//将第一个字节保存下来??
- ??????//将下一个起始符号之前的数据都保存下来??
- ??????while?(next4Bytes?!=?0x00000001?&&?(next4Bytes&0xFFFFFF00)?!=?0x00000100)?{??
- ????//?We?save?at?least?some?of?"next4Bytes".??
- ????if?((unsigned)(next4Bytes&0xFF)?>?1)?{??//一次可以保存4个字节,并不需要一个一个字节对比,除非到了结尾??
- ??????//?Common?case:?0x00000001?or?0x000001?definitely?doesn't?begin?anywhere?in?"next4Bytes",?so?we?save?all?of?it:??
- ??????save4Bytes(next4Bytes);??
- ??????skipBytes(4);??
- ??????//?Save?the?first?byte,?and?continue?testing?the?rest:??
- ??????saveByte(next4Bytes>>24);??
- ??????skipBytes(1);??
- ????next4Bytes?=?test4Bytes();??
- ??????//?Assert:?next4Bytes?starts?with?0x00000001?or?0x000001,?and?we've?saved?all?previous?bytes?(forming?a?complete?NAL?unit).??
- ??????//?Skip?over?these?remaining?bytes,?up?until?the?start?of?the?next?NAL?unit:??
- ??????if?(next4Bytes?==?0x00000001)?{??
- ????skipBytes(4);??
- ????skipBytes(3);??
- ???
- ????u_int8_t?nal_ref_idc?=?(firstByte&0x60)>>5;??
- ????u_int8_t?nal_unit_type?=?firstByte&0x1F;??
- ????????curFrameSize()-fOutputStartCodeSize,?nal_ref_idc,?nal_unit_type,?nal_unit_type_description[nal_unit_type]);??
- ????//下面根据NALU的类型来作具体的分析??
- ????switch?(nal_unit_type)?{??
- ??????case?6:?{?//?Supplemental?enhancement?information?(SEI)??
- ????analyze_sei_data();??
- ????//?Later,?perhaps?adjust?"fPresentationTime"?if?we?saw?a?"pic_timing"?SEI?payload????#####??
- ????break;??
- ??????case?7:?{?//?Sequence?parameter?set?(序列参数集)??
- ??????//??
- ??????//保存一份SPS的副本到H264VideoStreamFramer中,后面的pps也需要保存,sps中可能还包含了帧率信息??
- ????//?First,?save?a?copy?of?this?NAL?unit,?in?case?the?downstream?object?wants?to?see?it:??
- ????usingSource()->saveCopyOfSPS(fStartOfFrame?+?fOutputStartCodeSize,?fTo?-?fStartOfFrame?-?fOutputStartCodeSize);??
- ????//?Parse?this?NAL?unit?to?check?whether?frame?rate?information?is?present:??
- ????unsigned?num_units_in_tick,?time_scale,?fixed_frame_rate_flag;??
- ????analyze_seq_parameter_set_data(num_units_in_tick,?fixed_frame_rate_flag);??
- ????if?(time_scale?>?0?&&?num_units_in_tick?>?0)?{??
- ??????usingSource()->fFrameRate?=?time_scale/(2.0*num_units_in_tick);???//sps中包含了帧率信息??
-
- ??????if?(fixed_frame_rate_flag?==?0)?{??
- ????????fprintf(stderr,?"\tWARNING:?\"fixed_frame_rate_flag\"?was?not?set\n");??
- ????}?else?{????//sps中不包含帧率信息,则使用source中设置的默认帧率??
-
- ??????case?8:?{?//?Picture?parameter?set?(图像参数集PPS)??
- ????//?Save?a?copy?of?this?NAL?unit,?in?case?the?downstream?object?wants?to?see?it:??
- ????usingSource()->saveCopyOfPPS(fStartOfFrame?+?fOutputStartCodeSize,?fTo?-?fStartOfFrame?-?fOutputStartCodeSize);??
- ??????
- ????usingSource()->setPresentationTime();???//设置当前的时间??
- ????unsigned?long?secs?=?(unsigned?long)usingSource()->fPresentationTime.tv_sec;??
- ????unsigned?uSecs?=?(unsigned)usingSource()->fPresentationTime.tv_usec;??
-
- ????//如果这个NALU是一个VCL?NALU(即包含的是视频数据),我们需要扫描下一个NALU的起始符,??
- ????//以判断这个NALU是否是当前的"access?unit"(这个"access?unit"应该可以理解为一帧图像帧吧)。??
- ????//我们需要根据这个信息去指明何时该增加"fPresentationTime"(RTP?打包时也需要根据这个信息,决定是否设置"M"位)。??
- ????//?If?this?NAL?unit?is?a?VCL?NAL?unit,?we?also?scan?the?start?of?the?next?NAL?unit,?to?determine?whether?this?NAL?unit??
- ????//?ends?the?current?'access?unit'.??We?need?this?information?to?figure?out?when?to?increment?"fPresentationTime".??
- ????//?(RTP?streamers?also?need?to?know?this?in?order?to?figure?out?whether?or?not?to?set?the?"M"?bit.)??
- ????Boolean?thisNALUnitEndsAccessUnit?=?False;?//?until?we?learn?otherwise???
- ????if?(haveSeenEOF())?{??
- ??????//?There?is?no?next?NAL?unit,?so?we?assume?that?this?one?ends?the?current?'access?unit':??
- ??????thisNALUnitEndsAccessUnit?=?True;??
- ??????Boolean?const?isVCL?=?nal_unit_type?<=?5?&&?nal_unit_type?>?0;?//?Would?need?to?include?type?20?for?SVC?and?MVC?#####??
- ??????if?(isVCL)?{??
- ????u_int32_t?first4BytesOfNextNALUnit?=?test4Bytes();??
- ????u_int8_t?firstByteOfNextNALUnit?=?first4BytesOfNextNALUnit>>24;??
- ????u_int8_t?next_nal_ref_idc?=?(firstByteOfNextNALUnit&0x60)>>5;??
- ????u_int8_t?next_nal_unit_type?=?firstByteOfNextNALUnit&0x1F;??
- ????if?(next_nal_unit_type?>=?6)?{??
- ????????//下一个NALU不是VCL的,当前的"access?unit"结束了??
- ??????//?The?next?NAL?unit?is?not?a?VCL;?therefore,?we?assume?that?this?NAL?unit?ends?the?current?'access?unit':??
-
- ????????//下一个NALU也是VCL的,还需要检查一下它们是不是属于同一个"access?unit"??
- ??????//?The?next?NAL?unit?is?also?a?VLC.??We?need?to?examine?it?a?little?to?figure?out?if?it's?a?different?'access?unit'.??
- ??????//?(We?use?many?of?the?criteria?described?in?section?7.4.1.2.4?of?the?H.264?specification.)??
- ??????Boolean?IdrPicFlag?=?nal_unit_type?==?5;??
- ??????Boolean?next_IdrPicFlag?=?next_nal_unit_type?==?5;??
- ??????if?(next_IdrPicFlag?!=?IdrPicFlag)?{??
- ????????//?IdrPicFlag?differs?in?value??
-
- ????????thisNALUnitEndsAccessUnit?=?True;??
- ??????}?else?if?(next_nal_ref_idc?!=?nal_ref_idc?&&?next_nal_ref_idc*nal_ref_idc?==?0)?{??
- ????????//?nal_ref_idc?differs?in?value?with?one?of?the?nal_ref_idc?values?being?equal?to?0??
-
- ??????}?else?if?((nal_unit_type?==?1?||?nal_unit_type?==?2?||?nal_unit_type?==?5)??
- ?????????????&&?(next_nal_unit_type?==?1?||?next_nal_unit_type?==?2?||?next_nal_unit_type?==?5))?{??
- ????????//?Both?this?and?the?next?NAL?units?begin?with?a?"slice_header".??
- ????????//?Parse?this?(for?each),?to?get?parameters?that?we?can?compare:??
- ??????????
- ????????//?Current?NAL?unit's?"slice_header":??
- ????????unsigned?frame_num,?pic_parameter_set_id,?idr_pic_id;??
- ????????Boolean?field_pic_flag,?bottom_field_flag;??
- ????????analyze_slice_header(fStartOfFrame?+?fOutputStartCodeSize,?fTo,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ?????????????????frame_num,?idr_pic_id,?field_pic_flag,?bottom_field_flag);??
- ????????//?Next?NAL?unit's?"slice_header":??
-
- ????????u_int8_t?next_slice_header[NUM_NEXT_SLICE_HEADER_BYTES_TO_ANALYZE];??
- ????????testBytes(next_slice_header,?sizeof?next_slice_header);??
- ????????unsigned?next_frame_num,?next_pic_parameter_set_id,?next_idr_pic_id;??
- ????????Boolean?next_field_pic_flag,?next_bottom_field_flag;??
- ????????analyze_slice_header(next_slice_header,?&next_slice_header[sizeof?next_slice_header],?next_nal_unit_type,153); list-style:decimal-leading-zero outside; line-height:18px; margin:0px!important; padding:0px 3px 0px 10px!important"> ?????????????????next_frame_num,?next_idr_pic_id,?next_field_pic_flag,?next_bottom_field_flag);??
- ??????????
- ????????if?(next_frame_num?!=?frame_num)?{??
- ??????????//?frame_num?differs?in?value??
- ??????????fprintf(stderr,?"\t(frame_num?differs?in?value)\n");??
- ??????????thisNALUnitEndsAccessUnit?=?True;??
- ????????}?else?if?(next_pic_parameter_set_id?!=?pic_parameter_set_id)?{??
- ??????????//?pic_parameter_set_id?differs?in?value??
-
- ????????}?else?if?(next_field_pic_flag?!=?field_pic_flag)?{??
- ??????????//?field_pic_flag?differs?in?value??
-
- ????????}?else?if?(next_bottom_field_flag?!=?bottom_field_flag)?{??
- ??????????//?bottom_field_flag?differs?in?value??
-
- ????????}?else?if?(next_IdrPicFlag?==?1?&&?next_idr_pic_id?!=?idr_pic_id)?{??
- ??????????//?IdrPicFlag?is?equal?to?1?for?both?and?idr_pic_id?differs?in?value??
- ??????????//?Note:?We?already?know?that?IdrPicFlag?is?the?same?for?both.??
-
- ??????????thisNALUnitEndsAccessUnit?=?True;??
- ????????}??
- ????if?(thisNALUnitEndsAccessUnit)?{??
-
- ??????usingSource()->fPictureEndMarker?=?True;??//这里就是设置RTP打包时用到的M标志了??
- ??????++usingSource()->fPictureCount;??
- ??????//下一个NALU不再属于当前"access unit""时,才改变时间??
- ??????//?Note?that?the?presentation?time?for?the?next?NAL?unit?will?be?different:??
- ??????struct?timeval&?nextPT?=?usingSource()->fNextPresentationTime;?//?alias 这里是引用 ??
- ??????nextPT?=?usingSource()->fPresentationTime;??
- ??????double?nextFraction?=?nextPT.tv_usec/1000000.0?+?1/usingSource()->fFrameRate;???
- ??????unsigned?nextSecsIncrement?=?(long)nextFraction;??
- ??????nextPT.tv_sec?+=?(long)nextSecsIncrement;??
- ??????nextPT.tv_usec?=?(long)((nextFraction?-?nextSecsIncrement)*1000000);??
- ????setParseState();??
- ????return?curFrameSize();??
- ??}?catch?(int?/*e*/)?{??
-
- ????return?0;??//?the?parsing?got?interrupted??
- }??
(编辑:ASP站长网)
|