TIdHTTP Lightstreamer Client

[rlebeau]

Code:

pHTTP->Request->CustomHeaders->Add("Accept: */*");
pHTTP->Request->CustomHeaders->Add("Content-Type: application/x-www-form-urlencoded");
pHTTP->Request->CustomHeaders->Add("Connection: Keep-Alive");

You should not be using the CustomHeaders property for those particular headers.  The Request object has individual properties for them instead:

Code:

pHTTP->Request->Accept = "*/*";
pHTTP->Request->ContentType = "application/x-www-form-urlencoded";
pHTTP->Request->Connection = "Keep-Alive";

Code:

UnicodeString params, url = L"http://push.lightstreamer.com/lightstreamer/create_session.txt";
params.sprintf(L"LS_op2=create&LS_cid=mgQkwtwdysogQz2BJ4Ji+kOj2Bg&LS_adapter_set=DEMO&LS_protocol=TLCP-2.1.0";
// parametros
params = TIdURI::ParamsEncode(params, enUTF8);
// conexão
pSSParams = new TStringStream(params);
pHTTP->Post(url, pSSParams, pClienteLS->RcvdStream);

When sending an "application/x-www-form-urlencoded" post, you should be using a TStringList for the post fields. Let TIdHTTP handle encoding them for you (especially since ParamsEncode() is not the correct thing to use in this context anyway):

Code:

TStringList *params = new TStringList;
params->Add(L"LS_op2=create");
params->Add(L"LS_cid=mgQkwtwdysogQz2BJ4Ji+kOj2Bg");
params->Add(L"LS_adapter_set=DEMO");
params->Add(L"LS_protocol=TLCP-2.1.0");
pHTTP->Post(url, params, pClienteLS->RcvdStream);
delete params;

When SessionId arrives, I subscribe some stocks using another TIdHTTP and stock data becomes to arrive.

Session IDs are (usually) exchanged back and forth between client and server using HTTP cookies.  Are the two TIdHTTP objects sharing a single TIdCookieManager object?  If not, then the second TIdHTTP won't be able to send back cookies received by the first TIdHTTP.  TIdHTTP instances do not share cookies with each other by default, you have to set that up manually.

Server is sending data all the time, but TIdHTTP connects and receives some data, then stops for a few minutes, then another block of data arrives, then stops again, then receive another block, a lot of times. 

Sounds like the server is using server-side pushes to deliver streaming live data.  TIdHTTP is not designed to handle streaming data, though there are some workarounds for that (like using the TIdHTTP.OnChunkReceived event or hoNoReadChunked flag if the server is sending the pushes using HTTP chunking, or by using a custom TStream class with overwritten Write() method).  Otherwise, you may have to resort to using TIdTCPClient instead, manually implementing enough of the HTTP protocol to get the streaming started, and then you can read the data however you want.

It seems like to be filling a buffer, precisely a 32k (32768 bytes) buffer.

Yes, that is possible.  TIdCustomHTTP.ReadResult() calls TIdIOHandler.ReadStream() to read data, which uses a local buffer (which is 32K by default, but can be customized via the TIdIOHandler.RecvBufferSize property) to receive data from the socket before writing it to the TStream.  ReadStream() calls TIdIOHandler.ReadBytes() to fill that local buffer, and ReadBytes() waits for the requested number of bytes to fill the buffer before exiting.

Which, under normal conditions, should be perfectly fine.  HTTP has a structure to it.  Unless the HTTP response body uses an arbitrary data type that is terminated only by closing the socket, or the response body uses a MIME type (like 'multipart/x-mixed-replace') without being wrapped in HTTP chunking, then the response data has known sizes up front, which buffering takes into account.

What i'm doing wrong ?

Nothing.  This is just the way TIdHTTP works.

If the response has a 'Transfer-Encoding: chunked' header, the data is read in complete chunks (each chunk has its own size, and each chunk is read in blocks of RecvBufferSize up to that size) until the last chunk is received.  Only complete chucks are passed to the TIdHTTP.OnChunkReceived event and written to the target TStream.

Otherwise, if the response has a 'Content-Length' header, the data is read in blocks of RecvBufferSize and written to the TStream until the specified number of bytes have been received.

Otherwise, if the data has a 'Content-Type: multipart/...' header, the response is read in arbitrary blocks based on available data from the socket (and some of those blocks may themselves be read in blocks of RecvBufferSize) and written to the target TStream until the terminating MIME boundary has been received.

Otherwise, the data is read in blocks of RecvBufferSize and written to the TStream until the socket is disconnected.

Why received data does not reaches the stream Write method immediately ?

Because TIdIOHandler.ReadBytes() is being told by TIdHTTP to wait for X number of bytes to arrive (up to 32K at a time), so that is what it is doing.  TIdHTTP is not asking the socket "how many bytes do you have right now? X? OK, give me X bytes" (except MAYBE in the MIME w/o chunking scenario). It is (usuallly) telling the socket "OK, I want X bytes, so wait until X bytes are available and then give them to me".