1.netty能做什么
首先netty是一款高性能、封裝性良好且靈活、基于NIO(真·非阻塞IO)的開源框架。可以用來(lái)手寫web服務(wù)器、TCP服務(wù)器等,支持的協(xié)議豐富,如:常用的HTTP/HTTPS/WEBSOCKET,并且提供的大量的方法,十分靈活,可以根據(jù)自己的需求量身DIV一款服務(wù)器。
用netty編寫TCP的服務(wù)器/客戶端
1.可以自己設(shè)計(jì)數(shù)據(jù)傳輸協(xié)議如下面這樣:
2.可以自定義編碼規(guī)則和解碼規(guī)則
3.可以自定義客戶端與服務(wù)端的數(shù)據(jù)交互細(xì)節(jié),處理socket流攻擊、TCP的粘包和拆包問(wèn)題
2.Quick Start
創(chuàng)建一個(gè)普通的maven項(xiàng)目,不依賴任何的三方web服務(wù)器,用main方法執(zhí)行即可。
加入POM依賴
io.netty netty-all 4.1.6.Final com.fasterxml.jackson.core jackson-databind 2.9.7 log4j log4j 1.2.17
設(shè)計(jì)一套基于TCP的數(shù)據(jù)傳輸協(xié)議
publicclassTcpProtocol{ privatebyteheader=0x58; privateintlen; privatebyte[]data; privatebytetail=0x63; publicbytegetTail(){ returntail; } publicvoidsetTail(bytetail){ this.tail=tail; } publicTcpProtocol(intlen,byte[]data){ this.len=len; this.data=data; } publicTcpProtocol(){ } publicbytegetHeader(){ returnheader; } publicvoidsetHeader(byteheader){ this.header=header; } publicintgetLen(){ returnlen; } publicvoidsetLen(intlen){ this.len=len; } publicbyte[]getData(){ returndata; } publicvoidsetData(byte[]data){ this.data=data; } }
這里使用16進(jìn)制表示協(xié)議的開始位和結(jié)束位,其中0x58代表開始,0x63代表結(jié)束,均用一個(gè)字節(jié)來(lái)進(jìn)行表示。
TCP服務(wù)器的啟動(dòng)類
publicclassTcpServer{ privateintport; privateLoggerlogger=Logger.getLogger(this.getClass()); publicvoidinit(){ logger.info("正在啟動(dòng)tcp服務(wù)器……"); NioEventLoopGroupboss=newNioEventLoopGroup();//主線程組 NioEventLoopGroupwork=newNioEventLoopGroup();//工作線程組 try{ ServerBootstrapbootstrap=newServerBootstrap();//引導(dǎo)對(duì)象 bootstrap.group(boss,work);//配置工作線程組 bootstrap.channel(NioServerSocketChannel.class);//配置為NIO的socket通道 bootstrap.childHandler(newChannelInitializer(){ protectedvoidinitChannel(SocketChannelch)throwsException{//綁定通道參數(shù) ch.pipeline().addLast("logging",newLoggingHandler("DEBUG"));//設(shè)置log監(jiān)聽器,并且日志級(jí)別為debug,方便觀察運(yùn)行流程 ch.pipeline().addLast("encode",newEncoderHandler());//編碼器。發(fā)送消息時(shí)候用過(guò) ch.pipeline().addLast("decode",newDecoderHandler());//解碼器,接收消息時(shí)候用 ch.pipeline().addLast("handler",newBusinessHandler());//業(yè)務(wù)處理類,最終的消息會(huì)在這個(gè)handler中進(jìn)行業(yè)務(wù)處理 } }); bootstrap.option(ChannelOption.SO_BACKLOG,1024);//緩沖區(qū) bootstrap.childOption(ChannelOption.SO_KEEPALIVE,true);//ChannelOption對(duì)象設(shè)置TCP套接字的參數(shù),非必須步驟 ChannelFuturefuture=bootstrap.bind(port).sync();//使用了Future來(lái)啟動(dòng)線程,并綁定了端口 logger.info("啟動(dòng)tcp服務(wù)器啟動(dòng)成功,正在監(jiān)聽端口:"+port); future.channel().closeFuture().sync();//以異步的方式關(guān)閉端口 }catch(InterruptedExceptione){ logger.info("啟動(dòng)出現(xiàn)異常:"+e); }finally{ work.shutdownGracefully(); boss.shutdownGracefully();//出現(xiàn)異常后,關(guān)閉線程組 logger.info("tcp服務(wù)器已經(jīng)關(guān)閉"); } } publicstaticvoidmain(String[]args){ newTcpServer(8777).init(); } publicTcpServer(intport){ this.port=port; } }
只要是基于netty的服務(wù)器,都會(huì)用到bootstrap 并用這個(gè)對(duì)象綁定工作線程組,channel的Class,以及用戶DIV的各種pipeline的handler類,注意在添加自定義handler的時(shí)候,數(shù)據(jù)的流動(dòng)順序和pipeline中添加hanlder的順序是一致的。也就是說(shuō),從上往下應(yīng)該為:底層字節(jié)流的解碼/編碼handler、業(yè)務(wù)處理handler。
編碼器
編碼器是服務(wù)器按照協(xié)議格式返回?cái)?shù)據(jù)給客戶端時(shí)候調(diào)用的,繼承MessageToByteEncoder代碼:
publicclassEncoderHandlerextendsMessageToByteEncoder{ privateLoggerlogger=Logger.getLogger(this.getClass()); protectedvoidencode(ChannelHandlerContextctx,Objectmsg,ByteBufout)throwsException{ if(msginstanceofTcpProtocol){ TcpProtocolprotocol=(TcpProtocol)msg; out.writeByte(protocol.getHeader()); out.writeInt(protocol.getLen()); out.writeBytes(protocol.getData()); out.writeByte(protocol.getTail()); logger.debug("數(shù)據(jù)編碼成功:"+out); }else{ logger.info("不支持的數(shù)據(jù)協(xié)議:"+msg.getClass()+" 期待的數(shù)據(jù)協(xié)議類是:"+TcpProtocol.class); } } }
解碼器
解碼器屬于比較核心的部分,自定義解碼協(xié)議、粘包、拆包等都在里面實(shí)現(xiàn),繼承自ByteToMessageDecoder,其實(shí)ByteToMessageDecoder的內(nèi)部已經(jīng)幫我們處理好了拆包/粘包的問(wèn)題,只需要按照它的設(shè)計(jì)原則去實(shí)現(xiàn)decode方法即可:
publicclassDecoderHandlerextendsByteToMessageDecoder{ //最小的數(shù)據(jù)長(zhǎng)度:開頭標(biāo)準(zhǔn)位1字節(jié) privatestaticintMIN_DATA_LEN=6; //數(shù)據(jù)解碼協(xié)議的開始標(biāo)志 privatestaticbytePROTOCOL_HEADER=0x58; //數(shù)據(jù)解碼協(xié)議的結(jié)束標(biāo)志 privatestaticbytePROTOCOL_TAIL=0x63; privateLoggerlogger=Logger.getLogger(this.getClass()); protectedvoiddecode(ChannelHandlerContextctx,ByteBufin,List
首先是黏包問(wèn)題:
如圖,正常的數(shù)據(jù)傳輸應(yīng)該是像數(shù)據(jù)A那樣,一包就是一個(gè)完整的數(shù)據(jù),但也有不正常的情況,比如一包數(shù)據(jù)包含多個(gè)數(shù)據(jù)。而在ByteToMessageDecoder會(huì)默認(rèn)把二進(jìn)制的字節(jié)碼放在byteBuf中,因此我們?cè)赾ode的時(shí)候要知道會(huì)有這樣的場(chǎng)景。
而粘包問(wèn)題實(shí)際上不需要我們?nèi)ソ鉀Q,下面是ByteToMessageDecoder的源碼,callDecode中回調(diào)我們手寫解碼器的decode方法。
protectedvoidcallDecode(ChannelHandlerContextctx,ByteBufin,List
綜合上面的源碼分析后,我們發(fā)現(xiàn):decode方法在while循環(huán)中,也就是bytebuf只要有內(nèi)容就會(huì)一直調(diào)用decode方法進(jìn)行解碼操作,因此在解決粘包問(wèn)題時(shí),只需要按照正常流程來(lái)就行了,解析協(xié)議開頭、數(shù)據(jù)字節(jié)、結(jié)束標(biāo)志后將數(shù)據(jù)放入到out這個(gè)list中即可。后面將會(huì)有數(shù)據(jù)進(jìn)行粘包測(cè)試。
拆包問(wèn)題
有時(shí)候,我們接收到的數(shù)據(jù)是不完整的,一個(gè)包的數(shù)據(jù)被拆成了很多份被后再發(fā)送出去。這種情況有可能是數(shù)據(jù)太大,被分割成很多份發(fā)送出去。比如數(shù)據(jù)包B被拆成兩份進(jìn)行發(fā)送:
拆包問(wèn)題,同樣在ByteToMessageDecoder 給我們解決了,我們只需要按照netty的設(shè)計(jì)原則去寫decode代碼即可。
首先,假設(shè)需要我們自己去解決拆包問(wèn)題應(yīng)該怎么實(shí)現(xiàn)?
先從問(wèn)題開始分析,需要的是數(shù)據(jù)B,但是卻只收到了數(shù)據(jù)B_1,這個(gè)時(shí)候應(yīng)該等待剩余的數(shù)據(jù)B_2的到來(lái),收到的數(shù)據(jù)B_1應(yīng)該用一個(gè)累加器存起來(lái),等到B_2到來(lái)的時(shí)候?qū)砂鼣?shù)據(jù)合并起來(lái)再進(jìn)行解碼。
那么問(wèn)題是,如何讓ByteToMessageDecoder這個(gè)知道數(shù)據(jù)不完整呢,在DecoderHandler.decode中有這樣一段代碼:
if(len>=in.readableBytes()){ logger.debug(String.format("數(shù)據(jù)長(zhǎng)度不夠,數(shù)據(jù)協(xié)議len長(zhǎng)度為:%1$d,數(shù)據(jù)包實(shí)際可讀內(nèi)容為:%2$d正在等待處理拆包……",len,in.readableBytes())); in.resetReaderIndex(); /* **結(jié)束解碼,這種情況說(shuō)明數(shù)據(jù)沒(méi)有到齊,在父類ByteToMessageDecoder的callDecode中會(huì)對(duì)out和in進(jìn)行判斷 *如果in里面還有可讀內(nèi)容即in.isReadable為true,cumulation中的內(nèi)容會(huì)進(jìn)行保留,,直到下一次數(shù)據(jù)到來(lái),將兩幀的數(shù)據(jù)合并起來(lái),再解碼。 *以此解決拆包問(wèn)題 */ return; }
當(dāng)讀到協(xié)議中的len大于bytebuf的可讀內(nèi)容時(shí)候說(shuō)明數(shù)據(jù)不完整,發(fā)生了拆包,調(diào)用resetReaderIndex將讀操作指針復(fù)位,并結(jié)束方法。再看看父類中的CallDecode方法的一段代碼:
if(outSize==out.size()){//相等說(shuō)明,并沒(méi)有解析出來(lái)新的object到out中 if(oldInputLength==in.readableBytes()){//這里很重要,若相等說(shuō)明decode中沒(méi)有讀取任何內(nèi)容出來(lái),這里一般是發(fā)生拆包后,將ByteBuf的指針手動(dòng)重置。重置后從這個(gè)方法break出來(lái)。讓ByteToMessageDecoder去處理拆包問(wèn)題。這里就體現(xiàn)了要按照netty的設(shè)計(jì)原則來(lái)寫代碼 break;//退出該方法 }else{ continue;//這里直接continue,是考慮讓開發(fā)者去跳過(guò)某些字節(jié),比如收到了socket攻擊時(shí),數(shù)據(jù)不按照協(xié)議體來(lái)的時(shí)候,就直接跳過(guò)這些字節(jié) } }
退出callDecode后,返回到channelRead中:
publicvoidchannelRead(ChannelHandlerContextctx,Objectmsg)throwsException{ if(msginstanceofByteBuf){ CodecOutputListout=CodecOutputList.newInstance(); try{ ByteBufdata=(ByteBuf)msg; first=cumulation==null; if(first){ cumulation=data; }else{ cumulation=cumulator.cumulate(ctx.alloc(),cumulation,data); } callDecode(ctx,cumulation,out);//注意這里傳入的不是data,而是cumulator,這個(gè)對(duì)象相當(dāng)于一個(gè)累加器,也就是說(shuō)每次調(diào)用callDecode的時(shí)候傳入的byteBuf實(shí)際上是經(jīng)過(guò)累加后的cumulation }catch(DecoderExceptione){ throwe; }catch(Throwablet){ thrownewDecoderException(t); }finally{ if(cumulation!=null&&!cumulation.isReadable()){//這里若是數(shù)據(jù)被讀取完,會(huì)清空累加器cumulation numReads=0; cumulation.release(); cumulation=null; }elseif(++numReads>=discardAfterReads){ //WedidenoughreadsalreadytrytodiscardsomebytessowenotrisktoseeaOOME. //Seehttps://github.com/netty/netty/issues/4275 numReads=0; discardSomeReadBytes(); } intsize=out.size(); decodeWasNull=!out.insertSinceRecycled(); fireChannelRead(ctx,out,size); out.recycle(); } }else{ ctx.fireChannelRead(msg); } }
而channelRead方法是,收到一包數(shù)據(jù)后就會(huì)調(diào)用一次。至此,netty幫我們完美解決了拆包問(wèn)題。我們只需要按著他的設(shè)計(jì)原則:len>byteBuf.readableBytes時(shí)候,重置讀指針,結(jié)束decode即可。
業(yè)務(wù)處理handler類
這一層中數(shù)據(jù)已經(jīng)被完整的解析出來(lái)了,可以直接使用了:
publicclassBusinessHandlerextendsChannelInboundHandlerAdapter{ privateObjectMapperobjectMapper=ByteUtils.InstanceObjectMapper(); privateLoggerlogger=Logger.getLogger(this.getClass()); @Override publicvoidchannelRead(ChannelHandlerContextctx,Objectmsg)throwsException{ if(msginstanceofbyte[]){ logger.debug("解碼后的字節(jié)碼:"+newString((byte[])msg,"UTF-8")); try{ ObjectobjectContainer=objectMapper.readValue((byte[])msg,DTObject.class); if(objectContainerinstanceofDTObject){ DTObjectdata=(DTObject)objectContainer; if(data.getClassName()!=null&&data.getObject().length>0){ Objectobject=objectMapper.readValue(data.getObject(),Class.forName(data.getClassName())); logger.info("收到實(shí)體對(duì)象:"+object); } } }catch(Exceptione){ logger.info("對(duì)象反序列化出現(xiàn)問(wèn)題:"+e); } } } }
由于在decode中并沒(méi)有將字節(jié)碼反序列成對(duì)象,因此需要進(jìn)一步反序列化。在傳輸數(shù)據(jù)的時(shí)候,可能傳遞的對(duì)象不只是一種,因此在反序列化也要考慮到這一問(wèn)題。解決辦法是將傳輸?shù)膶?duì)象進(jìn)行二次包裝,將全名類信息包含進(jìn)去:
publicclassDTObject{ privateStringclassName; privatebyte[]object; }
這樣在反序列化的時(shí)候使用Class.forName()獲取Class,避免了要寫很多if循環(huán)判斷反序列化的對(duì)象的Class。前提是要類名和包路徑要完全匹配!
接下來(lái)編寫一個(gè)TCP客戶端進(jìn)行測(cè)試
啟動(dòng)類的init方法:
publicvoidinit()throwsInterruptedException{ NioEventLoopGroupgroup=newNioEventLoopGroup(); try{ Bootstrapbootstrap=newBootstrap(); bootstrap.group(group); bootstrap.channel(NioSocketChannel.class); bootstrap.option(ChannelOption.SO_KEEPALIVE,true); bootstrap.handler(newChannelInitializer(){ @Override protectedvoidinitChannel(Channelch)throwsException{ ch.pipeline().addLast("logging",newLoggingHandler("DEBUG")); ch.pipeline().addLast(newEncoderHandler()); ch.pipeline().addLast(newEchoHandler()); } }); bootstrap.remoteAddress(ip,port); ChannelFuturefuture=bootstrap.connect().sync(); future.channel().closeFuture().sync(); }catch(InterruptedExceptione){ e.printStackTrace(); }finally{ group.shutdownGracefully().sync(); } }
客戶端的handler:
publicclassEchoHandlerextendsChannelInboundHandlerAdapter{ //連接成功后發(fā)送消息測(cè)試 @Override publicvoidchannelActive(ChannelHandlerContextctx)throwsException{ Useruser=newUser(); user.setBirthday(newDate()); user.setUID(UUID.randomUUID().toString()); user.setName("冉鵬峰"); user.setAge(22); DTObjectdtObject=newDTObject(); dtObject.setClassName(user.getClass().getName()); dtObject.setObject(ByteUtils.InstanceObjectMapper().writeValueAsBytes(user)); TcpProtocoltcpProtocol=newTcpProtocol(); byte[]objectBytes=ByteUtils.InstanceObjectMapper().writeValueAsBytes(dtObject); tcpProtocol.setLen(objectBytes.length); tcpProtocol.setData(objectBytes); ctx.write(tcpProtocol); ctx.flush(); } }
這個(gè)handler是為了模擬在TCP連接建立好之后發(fā)送一包的數(shù)據(jù)到服務(wù)端經(jīng)行測(cè)試,通過(guò)channel的write去發(fā)送數(shù)據(jù),只要在啟動(dòng)類TcpClient配置了編碼器的EncoderHandler,就可以直接將對(duì)象tcpProtocol傳進(jìn)去,它將在EncoderHandler的encode方法中被自動(dòng)轉(zhuǎn)換成字節(jié)碼放入bytebuf中。
正常數(shù)據(jù)傳輸測(cè)試:
結(jié)果:
2019-01-1416:30:34DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-1416:30:34DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-1416:30:34DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-1416:30:34DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA0OjMwOjE0IiwidWlkIjoiOGY0OTM0OGEtMWNmMy00ZTEyLWEzZTAtY2M1ZTJjZTkzMDdlIn0="} 2019-01-1416:30:34INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='8f49348a-1cf3-4e12-a3e0-cc5e2ce9307e',birthday=MonJan1404:30:00CST2019}
可以看到最終的實(shí)體對(duì)象User被成功的解析出來(lái)。
在debug模式下還會(huì)看到這樣的一個(gè)表格在控制臺(tái)輸出:
+-------------------------------------------------+ |0123456789abcdef| +--------+-------------------------------------------------+----------------+ |00000000|58000000b57b22636c6173734e616d65|X....{"className| |00000010|223a22706f6a6f2e55736572222c226f|":"pojo.User","o| |00000020|626a656374223a2265794a755957316c|bject":"eyJuYW1l| |00000030|496a6f693559614a36626d5035624f77|Ijoi5YaJ6bmP5bOw| |00000040|496977695957646c496a6f794e437769|IiwiYWdlIjoyNCwi| |00000050|596d6c796447686b59586b694f694979|YmlydGhkYXkiOiIy| |00000060|4d4445354c7a41784c7a453049444130|MDE5LzAxLzE0IDA0| |00000070|4f6a4d774f6a45304969776964576c6b|OjMwOjE0IiwidWlk| |00000080|496a6f694f4759304f544d304f474574|IjoiOGY0OTM0OGEt| |00000090|4d574e6d4d7930305a5445794c57457a|MWNmMy00ZTEyLWEz| |000000a0|5a54417459324d315a544a6a5a546b7a|ZTAtY2M1ZTJjZTkz| |000000b0|4d44646c496e303d227d63|MDdlIn0="}c| +--------+-------------------------------------------------+----------------+
這個(gè)是相當(dāng)于真實(shí)的數(shù)據(jù)抓包展示,數(shù)據(jù)被轉(zhuǎn)換成字節(jié)碼后是以二進(jìn)制的形式在TCP緩存區(qū)沖傳輸過(guò)來(lái)。但是二進(jìn)制太長(zhǎng)了,所以一般都是轉(zhuǎn)換成16進(jìn)制顯示的,一個(gè)表格顯示一個(gè)字節(jié)的數(shù)據(jù),數(shù)據(jù)由地位到高位由左到右,由上到下進(jìn)行排列。
其中0x58為TcpProtocol中設(shè)置的開始標(biāo)志,00 00 00 b5為數(shù)據(jù)的長(zhǎng)度,因?yàn)槭莍nt類型所以占用了四個(gè)字節(jié)從7b--7d內(nèi)容為要傳輸?shù)臄?shù)據(jù)內(nèi)容,結(jié)尾的0x63為TcpProtocol設(shè)置的結(jié)束標(biāo)志位。
粘包測(cè)試
為了模擬粘包,首先將啟動(dòng)類TcpClient中配置的編碼器的EncoderHandler注釋掉:
bootstrap.handler(newChannelInitializer(){ @Override protectedvoidinitChannel(Channelch)throwsException{ ch.pipeline().addLast("logging",newLoggingHandler("DEBUG")); //ch.pipeline().addLast(newEncoderHandler());因?yàn)樾枰赽yteBuf中手動(dòng)模擬粘包的場(chǎng)景 ch.pipeline().addLast(newEchoHandler()); } });
然后在發(fā)送的時(shí)候故意將三幀的數(shù)據(jù),放在一個(gè)包中就行發(fā)送,在EchoHanlder做如下修改:
publicvoidchannelActive(ChannelHandlerContextctx)throwsException{ Useruser=newUser(); user.setBirthday(newDate()); user.setUID(UUID.randomUUID().toString()); user.setName("冉鵬峰"); user.setAge(24); DTObjectdtObject=newDTObject(); dtObject.setClassName(user.getClass().getName()); dtObject.setObject(ByteUtils.InstanceObjectMapper().writeValueAsBytes(user)); TcpProtocoltcpProtocol=newTcpProtocol(); byte[]objectBytes=ByteUtils.InstanceObjectMapper().writeValueAsBytes(dtObject); tcpProtocol.setLen(objectBytes.length); tcpProtocol.setData(objectBytes); ByteBufbuffer=ctx.alloc().buffer(); buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(tcpProtocol.getData()); buffer.writeByte(tcpProtocol.getTail()); //模擬粘包的第二幀數(shù)據(jù) buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(tcpProtocol.getData()); buffer.writeByte(tcpProtocol.getTail()); //模擬粘包的第三幀數(shù)據(jù) buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(tcpProtocol.getData()); buffer.writeByte(tcpProtocol.getTail()); ctx.write(buffer); ctx.flush(); }
運(yùn)行結(jié)果:
2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1651 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA0OjQ0OjE0IiwidWlkIjoiODFkZTU5YWUtMzQ4Mi00ZDFhLWJjNDMtN2NjMTJmOTI1ZTUxIn0="} 2019-01-14 1651 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='81de59ae-3482-4d1a-bc43-7cc12f925e51',birthday=MonJan140400CST2019} 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1651 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA0OjQ0OjE0IiwidWlkIjoiODFkZTU5YWUtMzQ4Mi00ZDFhLWJjNDMtN2NjMTJmOTI1ZTUxIn0="} 2019-01-14 1651 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='81de59ae-3482-4d1a-bc43-7cc12f925e51',birthday=MonJan140400CST2019} 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141651DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1651 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA0OjQ0OjE0IiwidWlkIjoiODFkZTU5YWUtMzQ4Mi00ZDFhLWJjNDMtN2NjMTJmOTI1ZTUxIn0="} 2019-01-14 1651 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='81de59ae-3482-4d1a-bc43-7cc12f925e51',birthday=MonJan140400CST2019}
服務(wù)器成功解析出來(lái)了三幀的數(shù)據(jù),BusinessHandler的channelRead方法被調(diào)用了三次。
而抓到的數(shù)據(jù)包也確實(shí)是模擬的三幀數(shù)據(jù)黏在一個(gè)包中:
+-------------------------------------------------+ |0123456789abcdef| +--------+-------------------------------------------------+----------------+ |00000000|58000000b57b22636c6173734e616d65|X....{"className| |00000010|223a22706f6a6f2e55736572222c226f|":"pojo.User","o| |00000020|626a656374223a2265794a755957316c|bject":"eyJuYW1l| |00000030|496a6f693559614a36626d5035624f77|Ijoi5YaJ6bmP5bOw| |00000040|496977695957646c496a6f794e437769|IiwiYWdlIjoyNCwi| |00000050|596d6c796447686b59586b694f694979|YmlydGhkYXkiOiIy| |00000060|4d4445354c7a41784c7a453049444130|MDE5LzAxLzE0IDA0| |00000070|4f6a51304f6a45304969776964576c6b|OjQ0OjE0IiwidWlk| |00000080|496a6f694f44466b5a54553559575574|IjoiODFkZTU5YWUt| |00000090|4d7a51344d6930305a4446684c574a6a|MzQ4Mi00ZDFhLWJj| |000000a0|4e444d744e324e6a4d544a6d4f544931|NDMtN2NjMTJmOTI1| |000000b0|5a545578496e303d227d【63】58000000b5|ZTUxIn0="}cX....| |000000c0|7b22636c6173734e616d65223a22706f|{"className":"po| |000000d0|6a6f2e55736572222c226f626a656374|jo.User","object| |000000e0|223a2265794a755957316c496a6f6935|":"eyJuYW1lIjoi5| |000000f0|59614a36626d5035624f774969776959|YaJ6bmP5bOwIiwiY| |00000100|57646c496a6f794e437769596d6c7964|WdlIjoyNCwiYmlyd| |00000110|47686b59586b694f6949794d4445354c|GhkYXkiOiIyMDE5L| |00000120|7a41784c7a4530494441304f6a51304f|zAxLzE0IDA0OjQ0O| |00000130|6a45304969776964576c6b496a6f694f|jE0IiwidWlkIjoiO| |00000140|44466b5a545535595755744d7a51344d|DFkZTU5YWUtMzQ4M| |00000150|6930305a4446684c574a6a4e444d744e|i00ZDFhLWJjNDMtN| |00000160|324e6a4d544a6d4f5449315a54557849|2NjMTJmOTI1ZTUxI| |00000170|6e303d227d【63】58000000b57b22636c61|n0="}cX....{"cla| |00000180|73734e616d65223a22706f6a6f2e5573|ssName":"pojo.Us| |00000190|6572222c226f626a656374223a226579|er","object":"ey| |000001a0|4a755957316c496a6f693559614a3662|JuYW1lIjoi5YaJ6b| |000001b0|6d5035624f77496977695957646c496a|mP5bOwIiwiYWdlIj| |000001c0|6f794e437769596d6c796447686b5958|oyNCwiYmlydGhkYX| |000001d0|6b694f6949794d4445354c7a41784c7a|kiOiIyMDE5LzAxLz| |000001e0|4530494441304f6a51304f6a45304969|E0IDA0OjQ0OjE0Ii| |000001f0|776964576c6b496a6f694f44466b5a54|widWlkIjoiODFkZT| |00000200|5535595755744d7a51344d6930305a44|U5YWUtMzQ4Mi00ZD| |00000210|46684c574a6a4e444d744e324e6a4d54|FhLWJjNDMtN2NjMT| |00000220|4a6d4f5449315a545578496e303d227d|JmOTI1ZTUxIn0="}| |00000230|【63】|c| +--------+-------------------------------------------------+----------------+
可以看到確實(shí)存在三個(gè)尾巴【63】
在netty4.x版本中,粘包問(wèn)題確實(shí)被netty的ByteToMessageDecoder中的CallDecode方法中給處理掉了。
拆包問(wèn)題
這次還是將TcpClient中的編碼器EncoderHandler注釋掉,然后在EchoHandler的channelActive中模擬數(shù)據(jù)的拆包問(wèn)題:
publicvoidchannelActive(ChannelHandlerContextctx)throwsException{ Useruser=newUser(); user.setBirthday(newDate()); user.setUID(UUID.randomUUID().toString()); user.setName("冉鵬峰"); user.setAge(24); DTObjectdtObject=newDTObject(); dtObject.setClassName(user.getClass().getName()); dtObject.setObject(ByteUtils.InstanceObjectMapper().writeValueAsBytes(user)); TcpProtocoltcpProtocol=newTcpProtocol(); byte[]objectBytes=ByteUtils.InstanceObjectMapper().writeValueAsBytes(dtObject); tcpProtocol.setLen(objectBytes.length); tcpProtocol.setData(objectBytes); ByteBufbuffer=ctx.alloc().buffer(); buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(Arrays.copyOfRange(tcpProtocol.getData(),0,tcpProtocol.getLen()/2));//只發(fā)送二分之一的數(shù)據(jù)包 //模擬拆包 ctx.write(buffer); ctx.flush(); Thread.sleep(3000);//模擬網(wǎng)絡(luò)延時(shí) buffer=ctx.alloc().buffer(); buffer.writeBytes(Arrays.copyOfRange(tcpProtocol.getData(),tcpProtocol.getLen()/2,tcpProtocol.getLen()));//將剩下的二分之和尾巴發(fā)送過(guò)去 buffer.writeByte(tcpProtocol.getTail()); ctx.write(buffer); ctx.flush(); }
運(yùn)行結(jié)果:
首先是客戶端這邊:
2019-01-141733DEBUG[DEBUG][id:0x3b8cbbbb,L:/127.0.0.1:51138-R:/127.0.0.1:8777]WRITE:95B +-------------------------------------------------+ |0123456789abcdef| +--------+-------------------------------------------------+----------------+ |00000000|58000000b57b22636c6173734e616d65|X....{"className| |00000010|223a22706f6a6f2e55736572222c226f|":"pojo.User","o| |00000020|626a656374223a2265794a755957316c|bject":"eyJuYW1l| |00000030|496a6f693559614a36626d5035624f77|Ijoi5YaJ6bmP5bOw| |00000040|496977695957646c496a6f794e437769|IiwiYWdlIjoyNCwi| |00000050|596d6c796447686b59586b694f6949|YmlydGhkYXkiOiI| +--------+-------------------------------------------------+----------------+ 2019-01-141733DEBUG[DEBUG][id:0x3b8cbbbb,L:/127.0.0.1:51138-R:/127.0.0.1:8777]FLUSH 2019-01-141736DEBUG[DEBUG][id:0x3b8cbbbb,L:/127.0.0.1:51138-R:/127.0.0.1:8777]WRITE:92B +-------------------------------------------------+ |0123456789abcdef| +--------+-------------------------------------------------+----------------+ |00000000|794d4445354c7a41784c7a4530494441|yMDE5LzAxLzE0IDA| |00000010|314f6a41344f6a45304969776964576c|1OjA4OjE0IiwidWl| |00000020|6b496a6f694f5745795a6a49354d6d4d|kIjoiOWEyZjI5MmM| |00000030|744d6a4d354f4330305a6a6b774c5746|tMjM5OC00ZjkwLWF| |00000040|6b5a5759745a6d466c4e44457a5a6a55|kZWYtZmFlNDEzZjU| |00000050|354e324533496e303d227d63|5N2E3In0="}c| +--------+-------------------------------------------------+----------------+ 2019-01-141736DEBUG[DEBUG][id:0x3b8cbbbb,L:/127.0.0.1:51138-R:/127.0.0.1:8777]FLUSH
確實(shí)是將數(shù)據(jù)分成兩包發(fā)送出去了
再看看服務(wù)端的輸出日志:
2019-01-141733DEBUG[DEBUG][id:0x8e5811b3,L:/127.0.0.1:8777-R:/127.0.0.1:51138]RECEIVED:95B +-------------------------------------------------+ |0123456789abcdef| +--------+-------------------------------------------------+----------------+ |00000000|58000000b57b22636c6173734e616d65|X....{"className| |00000010|223a22706f6a6f2e55736572222c226f|":"pojo.User","o| |00000020|626a656374223a2265794a755957316c|bject":"eyJuYW1l| |00000030|496a6f693559614a36626d5035624f77|Ijoi5YaJ6bmP5bOw| |00000040|496977695957646c496a6f794e437769|IiwiYWdlIjoyNCwi| |00000050|596d6c796447686b59586b694f6949|YmlydGhkYXkiOiI| +--------+-------------------------------------------------+----------------+ 2019-01-141733DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141733DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-14 1733 DEBUG [org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)長(zhǎng)度不夠,數(shù)據(jù)協(xié)議len長(zhǎng)度為:181,數(shù)據(jù)包實(shí)際可讀內(nèi)容為:90正在等待處理拆包…… 2019-01-141736DEBUG[DEBUG][id:0x8e5811b3,L:/127.0.0.1:8777-R:/127.0.0.1:51138]RECEIVED:92B +-------------------------------------------------+ |0123456789abcdef| +--------+-------------------------------------------------+----------------+ |00000000|794d4445354c7a41784c7a4530494441|yMDE5LzAxLzE0IDA| |00000010|314f6a41344f6a45304969776964576c|1OjA4OjE0IiwidWl| |00000020|6b496a6f694f5745795a6a49354d6d4d|kIjoiOWEyZjI5MmM| |00000030|744d6a4d354f4330305a6a6b774c5746|tMjM5OC00ZjkwLWF| |00000040|6b5a5759745a6d466c4e44457a5a6a55|kZWYtZmFlNDEzZjU| |00000050|354e324533496e303d227d63|5N2E3In0="}c| +--------+-------------------------------------------------+----------------+ 2019-01-141736DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141736DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141736DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1736 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA1OjA4OjE0IiwidWlkIjoiOWEyZjI5MmMtMjM5OC00ZjkwLWFkZWYtZmFlNDEzZjU5N2E3In0="} 2019-01-14 1736 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='9a2f292c-2398-4f90-adef-fae413f597a7',birthday=MonJan140500CST2019}
在第一包數(shù)據(jù),判斷到bytebuf中的可讀內(nèi)容不夠的時(shí)候,終止解碼,并且從父類的callDecode中的while循環(huán)break出去,在父類的channelRead中等待下一包數(shù)據(jù)到來(lái)的時(shí)候?qū)砂鼣?shù)據(jù)合并起來(lái)再次decode解碼。
最后測(cè)試下同時(shí)出現(xiàn)拆包、粘包的場(chǎng)景
還是將TcpClient中的編碼器EncoderHandler注釋掉,然后在EchoHandler的ChannelActive方法:
publicvoidchannelActive(ChannelHandlerContextctx)throwsException{ Useruser=newUser(); user.setBirthday(newDate()); user.setUID(UUID.randomUUID().toString()); user.setName("冉鵬峰"); user.setAge(24); DTObjectdtObject=newDTObject(); dtObject.setClassName(user.getClass().getName()); dtObject.setObject(ByteUtils.InstanceObjectMapper().writeValueAsBytes(user)); TcpProtocoltcpProtocol=newTcpProtocol(); byte[]objectBytes=ByteUtils.InstanceObjectMapper().writeValueAsBytes(dtObject); tcpProtocol.setLen(objectBytes.length); tcpProtocol.setData(objectBytes); ByteBufbuffer=ctx.alloc().buffer(); buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(Arrays.copyOfRange(tcpProtocol.getData(),0,tcpProtocol.getLen()/2));//拆包,只發(fā)送一半的數(shù)據(jù) ctx.write(buffer); ctx.flush(); Thread.sleep(3000); buffer=ctx.alloc().buffer(); buffer.writeBytes(Arrays.copyOfRange(tcpProtocol.getData(),tcpProtocol.getLen()/2,tcpProtocol.getLen()));//拆包發(fā)送剩余的一半數(shù)據(jù) buffer.writeByte(tcpProtocol.getTail()); //模擬粘包的第二幀數(shù)據(jù) buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(tcpProtocol.getData()); buffer.writeByte(tcpProtocol.getTail()); //模擬粘包的第三幀數(shù)據(jù) buffer.writeByte(tcpProtocol.getHeader()); buffer.writeInt(tcpProtocol.getLen()); buffer.writeBytes(tcpProtocol.getData()); buffer.writeByte(tcpProtocol.getTail()); ctx.write(buffer); ctx.flush(); }
最后直接查看服務(wù)端的輸出結(jié)果:
2019-01-141725DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141725DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-14 1725 DEBUG [org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)長(zhǎng)度不夠,數(shù)據(jù)協(xié)議len長(zhǎng)度為:181,數(shù)據(jù)包實(shí)際可讀內(nèi)容為:90正在等待處理拆包…… 2019-01-141728DEBUG[DEBUG][id:0xc46234aa,L:/127.0.0.1:8777-R:/127.0.0.1:51466]RECEIVED:466B 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1728 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA1OjE5OjE0IiwidWlkIjoiODE2Zjg2ZDItNDBhMS00MDRkLTgwMWItZmY1NzgxMTJhNjFmIn0="} 2019-01-14 1728 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='816f86d2-40a1-404d-801b-ff578112a61f',birthday=MonJan140500CST2019} 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1728 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA1OjE5OjE0IiwidWlkIjoiODE2Zjg2ZDItNDBhMS00MDRkLTgwMWItZmY1NzgxMTJhNjFmIn0="} 2019-01-14 1728 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='816f86d2-40a1-404d-801b-ff578112a61f',birthday=MonJan140500CST2019} 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]開始解碼數(shù)據(jù)…… 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)開頭格式正確 2019-01-141728DEBUG[org.wisdom.server.decoder.DecoderHandler]數(shù)據(jù)解碼成功 2019-01-14 1728 DEBUG [org.wisdom.server.business.BusinessHandler]解碼后的字節(jié)碼:{"className":"pojo.User","object":"eyJuYW1lIjoi5YaJ6bmP5bOwIiwiYWdlIjoyNCwiYmlydGhkYXkiOiIyMDE5LzAxLzE0IDA1OjE5OjE0IiwidWlkIjoiODE2Zjg2ZDItNDBhMS00MDRkLTgwMWItZmY1NzgxMTJhNjFmIn0="} 2019-01-14 1728 INFO [org.wisdom.server.business.BusinessHandler]收到實(shí)體對(duì)象:User{name='冉鵬峰',age=24,UID='816f86d2-40a1-404d-801b-ff578112a61f',birthday=MonJan140500CST2019}
總結(jié)
對(duì)于拆包、粘包只要配合netty的設(shè)計(jì)原則去實(shí)現(xiàn)代碼,就能愉快且輕松的解決了。本例雖然通過(guò)DTObject包裝了數(shù)據(jù),避免解碼時(shí)每增加一種對(duì)象類型,就要新增一個(gè)if判斷的尷尬。但是仍然無(wú)法處理傳輸List、Map時(shí)候的場(chǎng)景。
審核編輯:湯梓紅
-
服務(wù)器
+關(guān)注
關(guān)注
12文章
9123瀏覽量
85324 -
JAVA
+關(guān)注
關(guān)注
19文章
2966瀏覽量
104702 -
TCP
+關(guān)注
關(guān)注
8文章
1353瀏覽量
79055
原文標(biāo)題:實(shí)現(xiàn)一款高可用的 TCP 數(shù)據(jù)傳輸服務(wù)器(Java版)
文章出處:【微信號(hào):AndroidPush,微信公眾號(hào):Android編程精選】歡迎添加關(guān)注!文章轉(zhuǎn)載請(qǐng)注明出處。
發(fā)布評(píng)論請(qǐng)先 登錄
相關(guān)推薦
評(píng)論