I have a simple application which connects to port 50000 on a Linux box, communicates, then disconnects. I'd like to log the entire communication in each direction in an easy-to-read format so that I can see what is going on. The protocol is ASCII but with a few control chars thrown in (eg STX/ETX/ACK/etc, and I wouldn't like to rule out NULL chars ever appearing in the data). I can easily run the server side at (say) port 50001 instead and have "something" listen on port 50000 and connect to port 50000 and pass the data back and forth between the two, logging it as it goes, although I don't know what to choose for "something". Or I could look at Wireshark, which I've played with in the past but never in anger, and my general feeling is that unless I learn how to sort the trees from the wood then this is going to give me too much information and I'm going to spend more time sorting through it than I do resolving the issues in the protocol. Or maybe there are better alternatives altogether! [If it's relevant: the Linux server has no GUI but is a VirtualBox virtual machine on another Linux server that does have a GUI where I could run Wireshark. The device connecting to the server is not a PC and has no diagnostic capability built in.] -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
Mark Use Wireshark and then go to Capture -> Options Ensure the correct interface is selected in the drop down. Because you are capturing on the same machine you dont need "Capture packets in promiscuous mode" ( but I dont think it will harm you. And in the field "Capture filter" enter "port 50000" then click "Start" and it will only capture stuff on the port. You then will only capture the info you require. There are other tools to "follow this stream" and I think this will put the conversation(s) back into "proper text". HTH Keith
Date: Wed, 30 Mar 2011 11:08:43 +0100 From: mark@quarella.co.uk To: main@lists.alug.org.uk Subject: [ALUG] TCP/IP logging
I have a simple application which connects to port 50000 on a Linux box, communicates, then disconnects.
I'd like to log the entire communication in each direction in an easy-to-read format so that I can see what is going on. The protocol is ASCII but with a few control chars thrown in (eg STX/ETX/ACK/etc, and I wouldn't like to rule out NULL chars ever appearing in the data).
I can easily run the server side at (say) port 50001 instead and have "something" listen on port 50000 and connect to port 50000 and pass the data back and forth between the two, logging it as it goes, although I don't know what to choose for "something".
Or I could look at Wireshark, which I've played with in the past but never in anger, and my general feeling is that unless I learn how to sort the trees from the wood then this is going to give me too much information and I'm going to spend more time sorting through it than I do resolving the issues in the protocol.
Or maybe there are better alternatives altogether!
[If it's relevant: the Linux server has no GUI but is a VirtualBox virtual machine on another Linux server that does have a GUI where I could run Wireshark. The device connecting to the server is not a PC and has no diagnostic capability built in.]
-- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
_______________________________________________ main@lists.alug.org.uk http://www.alug.org.uk/ http://lists.alug.org.uk/mailman/listinfo/main Unsubscribe? See message headers or the web site above!
On Wed, 30 Mar 2011 18:22:27 +0100 <keithjamieson@hotmail.co.uk> allegedly wrote:
Mark
Use Wireshark and then go to
Capture -> Options
Or of you don't want the overhead of using wireshark (or don't have a GUI) look at tcpdump and then use wireshark to analyse the capture file afterwards. Or you could probably use ngrep to capture those specific bits of the conversation you are interested in. Or you could take a look at netcat which you could fire up as a listener acting somewhat like a proxy with logging. (In fact I'm pretty sure that you should be able to find some scripts built around netcat which will do what you want.) HTH Mick --------------------------------------------------------------------- The text file for RFC 854 contains exactly 854 lines. Do you think there is any cosmic significance in this? Douglas E Comer - Internetworking with TCP/IP Volume 1 http://www.ietf.org/rfc/rfc854.txt ---------------------------------------------------------------------
On 30/03/11 19:07, mick wrote:
Or of you don't want the overhead of using wireshark (or don't have a GUI) look at tcpdump and then use wireshark to analyse the capture file afterwards.
I do need to play more with tcpdump but I really don't find Wireshark to be a convenient tool for analysis (I'm sure it's something I'm doing wrong!). All I actually want to see is something like: [2011-03-31 10:05:01.0001] >> 48 65 6c 6c 6f 0a |Hello.| [2011-03-31 10:05:01.0021] << 57 6f 72 6c 64 0a |World.| .. to show one end "saying" Hello and the other end "replying" World. Are there tools that would give me that from a pcap log, or ways to do it in Wireshark? It would help if I knew how to describe what I'm looking for without mocking up an example - Google's not much help when you don't know what to call what you're looking for!
Or you could probably use ngrep to capture those specific bits of the conversation you are interested in.
ngrep looks like fun, although when I tried it and generated some comms all I got was: $ sudo ngrep -x any port 50000 interface: eth0 (10.0.0.0/255.255.255.0) filter: (ip or ip6) and ( port 50000 ) match: any ##exit 6 received, 0 dropped I can play with examples of usage for HTTP etc, and they all work as expected, but all I get from my application is "##" or maybe "####". The data is binary and in small messages and I guess ngrep is waiting for enough data to show me, although if that's the case I can't see how to control it? -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
On 30/03/11 18:22, keithjamieson@hotmail.co.uk wrote:
Use Wireshark and then go to
Capture -> Options
Ensure the correct interface is selected in the drop down. Because you are capturing on the same machine you dont need "Capture packets in promiscuous mode" ( but I dont think it will harm you.
And in the field "Capture filter" enter "port 50000" then click "Start" and it will only capture stuff on the port.
Thanks, that's given me a Wireshark log. (And it looks like the problem I'm investigating is a firmware issue in the device I'm talking to, as the manufacturer has taken that log and think they've spotted the issue.)
You then will only capture the info you require. There are other tools to "follow this stream" and I think this will put the conversation(s) back into "proper text".
I think it's the "follow this stream" bit that I'm interested in now, I'll go and play. -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
On 30/03/11 18:22, keithjamieson@hotmail.co.uk wrote:
Use Wireshark and then go to
Capture -> Options
Ensure the correct interface is selected in the drop down. Because you are capturing on the same machine you dont need "Capture packets in promiscuous mode" ( but I dont think it will harm you.
And in the field "Capture filter" enter "port 50000" then click "Start" and it will only capture stuff on the port.
Thanks, that's given me a Wireshark log. (And it looks like the problem I'm investigating is a firmware issue in the device I'm talking to, as the manufacturer has taken that log and think they've spotted the issue.)
Good.
You then will only capture the info you require. There are other tools to "follow this stream" and I think this will put the conversation(s) back into "proper text".
I think it's the "follow this stream" bit that I'm interested in now, I'll go and play.
I only have SIP traces with me ATM and I click on "Follow UDP Stream" and this gives me the text of the SIP conversation. You have the options to view as various options, find, save as, and print. I dont know the format of what you are using, but glad you got enough info to blame them :) Keith
Update on this (desire to log a TCP/IP "conversation" but also being able to sort the signal from the noise): Over the weekend I read about tcpflow in Linux Magazine, and it looks like just what I needed (although I haven't had chance to play with it yet). Mark Rogers -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
I can easily run the server side at (say) port 50001 instead and have "something" listen on port 50000 and connect to port 50000 and pass the data back and forth between the two, logging it as it goes, although I don't know what to choose for "something".
Wouldn't the "something" be netcat and tee. I googled and came up with these: http://smaftoul.wordpress.com/2009/05/13/netcat-as-a-logging-tcp-proxy/ http://blog.hawkhost.com/2009/12/12/using-netcat-as-an-intercepting-proxy/ Hope that helps, Richard
On 31/03/11 09:10, Richard Parsons wrote:
Wouldn't the "something" be netcat and tee. I googled and came up with these:
http://smaftoul.wordpress.com/2009/05/13/netcat-as-a-logging-tcp-proxy/ http://blog.hawkhost.com/2009/12/12/using-netcat-as-an-intercepting-proxy/
The problem with these approaches is (as I understand them) I end up with two files (an incoming and an outgoing) with no way to "merge" them to see the sequence of the conversation? -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
On Thu, Mar 31, 2011 at 10:10:58AM +0100, Mark Rogers wrote:
On 31/03/11 09:10, Richard Parsons wrote:
Wouldn't the "something" be netcat and tee. I googled and came up with these:
http://smaftoul.wordpress.com/2009/05/13/netcat-as-a-logging-tcp-proxy/ http://blog.hawkhost.com/2009/12/12/using-netcat-as-an-intercepting-proxy/
The problem with these approaches is (as I understand them) I end up with two files (an incoming and an outgoing) with no way to "merge" them to see the sequence of the conversation?
Couldn't you just use the same filename for both tee commands? You could even echo something before each one so you knew which were in-bound and which were out-bound. Am I missing something?
On 31/03/11 12:14, Richard Parsons wrote:
Couldn't you just use the same filename for both tee commands? You could even echo something before each one so you knew which were in-bound and which were out-bound. Am I missing something?
To be honest I just couldn't see how that would work, but I'll try it and see. -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
On 31 March 2011 12:14, Richard Parsons <richard.lee.parsons@gmail.com> wrote:
On Thu, Mar 31, 2011 at 10:10:58AM +0100, Mark Rogers wrote:
On 31/03/11 09:10, Richard Parsons wrote:
Wouldn't the "something" be netcat and tee. I googled and came up with these:
http://smaftoul.wordpress.com/2009/05/13/netcat-as-a-logging-tcp-proxy/ http://blog.hawkhost.com/2009/12/12/using-netcat-as-an-intercepting-proxy/
The problem with these approaches is (as I understand them) I end up with two files (an incoming and an outgoing) with no way to "merge" them to see the sequence of the conversation?
Couldn't you just use the same filename for both tee commands? You could even echo something before each one so you knew which were in-bound and which were out-bound. Am I missing something?
How could you echo a marker in both incoming and outgoing case each time the data arrived? And what about timing of the data? Would you not want each incoming and outgoing data to have a timestamp so you can: - play it back in the correct sequence / order / speed - calculate delays Srdjan
On 03/04/11 11:29, Srdjan Todorovic wrote:
And what about timing of the data? Would you not want each incoming and outgoing data to have a timestamp so you can: - play it back in the correct sequence / order / speed - calculate delays
Just to say: "yes, that's what I want" I'm 90% sure a lot of the problems I am having are time dependant, ie events happening faster than the system can cope with them, possibly leading to corrupt messages from overflowing buffers. -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) @ 13 Clarke Rd, Milton Keynes, MK1 1LG
participants (5)
-
keithjamieson＠hotmail.co.uk -
Mark Rogers -
mick -
Richard Parsons -
Srdjan Todorovic