Fog Creek Software
Discussion Board




Welcome! and rules

Joel on Software

Streaming

Is it practical to "push" data continuously from a server component to a client using the remoting framework, or using COM+ (and if so, how do you do that)? Or is it necessary to use a socket-based stream for this?

I have an infinite-length stream of data (being captured from a device) to pass from the server to the client.

Christopher Wells
Friday, February 27, 2004

It's difficult to answer this question without knowing more about the way you need the data.

- Is timely delivery important? (i.e., listening to a stream)
- Is reliable delivery important?
- How close are you to filling the network pipe?

Brad Wilson (dotnetguy.techieswithcats.com)
Friday, February 27, 2004

Thanks Brad.

> It's difficult to answer this question without knowing more about the way you need the data.

I have a client application on one machine, and a device server on another, both coded in .NET.

There are several API functions between the two (open device, configure device, etc). Instead handling serialization and deserialization of these commands/responses over a sockets, I thought I'd try the .NET remoting framework.

In addition to this set of command/response messages, another type of communication is:

client: start test, stream me the captured data
server: (streams the data)
client: ok, stop streaming now

> Is timely delivery important? (i.e., listening to a stream)

I'm not sure I understand this question. On the client I'll be displaying the received data stream in close to real time ... after allowing for the buffering delays associated with TCP packetisation etc., I thought I could hope to display data with like a 1 second delay (so that bits captured on the server at time 'n' are displayed at time 'n+1' on the client).

> Is reliable delivery important?

Yes and no. The device server also writes its captured data (reliably) to a local disk file, which can be (reliably) retrieved later after the test is finished. The purpose of the real-time streaming is to monitor the test ... it would be OK (though not ideal) if some of those packets were lost.

So I can imagine two possible implementations for the real-time streaming:

* Use TCP (which guarantees delivery); the problems with TCP are that it may insert arbitrarily long delays (which I was hoping to avoid in practice by allowing for say an entire second between the client and the server), and that in the long term the volume of data may exceed the available bandwidth (so I might control the volume of data, either by sending less data by using a lower sample frequency at the device, or by discarding chunks of data whenever too much data is buffered in the server).

* Use UDP/RTP (which is better than TCP at guaranteeing timeliness, but doesn't guarantee delivery); the problems with UDP/RTP is that it doesn't guarantee delivery; it may not be supported by higher-level protocols like the .NET remoting framewok, if I'm hoping to use that instead of using sockets myself; and mechanisms to improve reliability (e.g. to recover transparently from the loss of any single packet, by sending each bit more than once using more than one packet) increases the total volume of data to be transfered and increases the complexity of the code.

> How close are you to filling the network pipe?

Let's assume I can control the volume of data being transfered: for example, I can reduce the sample frequency of the device, and/or reduce the number of 'signal channels' that are being transfered.

I have no control over the bandwidth of the network pipe. The likely configurations are:

* Client and server both running on the same machine
* Client and server connected via an unmanaged LAN
* In the not too distant future, two other configurations will be: client and server connecting via wireless; client and server connecting over the internet, perhaps with a dial-up connection on the server side.

Finally, if I can or should use the .NET remoting framework for this, I'm not sure how to: for example if I look at the sample program at the end of http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/hawkremoting.asp?frame=true would I be trying to use a System.IO.MemoryStream which exists simultaneously on the server and on the client (which the server will write to and which the client will read from)? For example, by modifying the object.cs to look like as follows:

namespace MyNamespace {
  public class StreamServer : MarshalByRefObject {

    public StreamServer() {
      Console.WriteLine("StreamServer activated");
    }

    public MemoryStream m_stream = new MemoryStream();

    public MemoryStream getStream() {
      //this method is called by the remote client ... return to the client an instance/reference of my stream
      return m_stream;
    }

    public writeToStream(byte[] buffer) {
      //this method is called from elsewhere in the device server ...it writes bytes to the MemoryStream ... the client can then read these bytes from its (remoted) copy of the stream
      m_stream.Write(buffer,0,buffer.Length);
    }
  }
}

Christopher Wells
Friday, February 27, 2004

I used Google to search microsoft.public.dotnet before asking this question here: it seems that a question like mine has been asked before, but not answered.

Anyway, I'm thinking that the remoting framework isn't appropriate for streaming; and that, given I'm considering TCP instead of UDP, I might use the "NetworkStream" class.

Christopher Wells
Saturday, February 28, 2004

*  Recent Topics

*  Fog Creek Home