Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Images streaming using Indy client side
#1
Hi,
I found this thread about images streaming from a camera and it was really good but I can't find any example on how to handle this on the client side for the Server push example:

https://stackoverflow.com/questions/2565...ttp-server

Any ideas on how to handle client side in C++ builder code not Delphi?
Reply
#2
(04-29-2022, 08:27 AM)Ahmed Sayed Wrote: Any ideas on how to handle client side in C++ builder code not Delphi?

The procedure is exactly the same in C++Builder as it is in Delphi.

If you are using an up-to-date version of Indy, you can use the hoNoReadMultipartMIME flag in the TIdHTTP::HTTPOptions property.

Enable the flag, and then call TIdHTTP::Get() to begin the streaming. When TIdHTTP.Get() exits, check to make sure that the response's ResponseCode is 200 and ContentType is "multipart/x-mixed-replace", and if so then you can manually read the server's pushed updates from the TIdHTTP::IOHandler as needed until disconnected.

You can use TIdMessageDecoderMIME to help you parse the incoming MIME data, for example:

Code:
IdHTTP1->HTTPOptions = IdHTTP1->HTTPOptions << hoNoReadMultipartMIME;
IdHTTP1->Get(_D("http://server/stream"));

if (IdHTTP1->ResponseCode != 200 || !IsHeaderMediaType(IdHTTP1->Response->ContentType, _D("multipart/x-mixed-replace"))) return;

String LBoundary = ExtractHeaderSubItem(IdHTTP1.Request->ContentType, _D("boundary"), QuoteHTTP);
// or: String LBoundary := IdHTTP1->Response->RawHeaders->Params[_D("Content-Type"), _D("boundary")];

String LBoundaryStart = _D("--") + LBoundary;
String LBoundaryEnd = LBoundaryStart + _D("--");

do
{
    String LLine = IdHTTP1->IOHandler->ReadLn();
    if (LLine == LBoundaryStart) break;
    if (LLine == LBoundaryEnd) return;
}
until (true);

TIdTCPStream *Stream = new TIdTCPStream(IdHTTP1);
try
{
    TIdMessageDecoder *LDecoder = new TIdMessageDecoderMIME(NULL);
    try
    {
        bool LMsgEnd = false;
        do
        {
            static_cast<TIdMessageDecoderMIME*>(LDecoder)->MIMEBoundary = LBoundary;
            LDecoder->SourceStream = Stream;
            LDecoder->FreeSourceStream = false;

            LDecoder->ReadHeader();
            switch (LDecoder->PartType())
            {
                case mcptText:
                case mcptAttachment:
                {
                    TMemoryStream *LMStream = new TMemoryStream;
                    try
                    {
                        TIdMessageDecoder *LNewDecoder = LDecoder->ReadBody(LMStream, LMsgEnd);
                        try
                        {
                            // process LMStream as needed...
                        }
                        catch (const Exception &)
                        {
                            delete LNewDecoder;
                            throw;
                        }
                        delete LDecoder;
                        LDecoder = LNewDecoder;
                    }
                    __finally
                    {
                        delete LMStream;
                    }
                    break;
                }

                case mcptIgnore:
                {
                    FreeAndNil(LDecoder);
                    LDecoder = new TIdMessageDecoderMIME(NULL);
                    break;
                }

                case mcptEOF:
                {
                    LMsgEnd = true;
                    break;
                }
            }
        }
        while (LDecoder && !LMsgEnd);
    }
    __finally
    {
        delete LDecoder;
    }
}
__finally
{
    delete Stream;
}

Reply
#3
Thanks I will test this example
Reply
#4
I have an issue with images being sent from server. I am using Berlin Upd2 and FMX camera component.

The data received on client side is bigger than the one sent from the server. I don't know why?

Server Side:
Camera buffering
Code:
void __fastcall TMain::CameraComponentSampleBufferReady(TObject *Sender, const TMediaTime ATime)
{
bmp->Clear(0);
CameraComponent->SampleBufferToBitmap(bmp.get(), true);
}
//---------------------------------------------------------------------------

Server get:
Code:
if (ARequestInfo->Document == "/stream")
    {
    if (ARequestInfo->Command == "GET")
        {
        IsStreaming = true;
        AResponseInfo->ResponseNo = 200;
        AResponseInfo->ContentType = "multipart/x-mixed-replace; boundary=imgboundary";
        AResponseInfo->CloseConnection = false;
        AResponseInfo->WriteHeader();

        TIdIOHandler* IOHandler = AContext->Connection->IOHandler;
        IOHandler->LargeStream = true;

        IOHandler->WriteLn("--imgboundary");

        do
            {
            if (!AContext->Connection->Connected())
                return;

            unique_ptr<TMemoryStream> str(new TMemoryStream);
            bmp->SaveToStream(str.get());
            str->Position = 0;
            IOHandler->WriteLn("Content-type: image/png");
            IOHandler->WriteLn();
            IOHandler->Write(str.get());
            IOHandler->WriteLn();      //When client disconnects this raises exception
            IOHandler->WriteLn("--imgboundary");

            Sleep(5000);

            } while(IsStreaming);

        return;
        }

Client Side:
Code:
void __fastcall TMain::ButtonClick(TObject *Sender)
{
IdHTTP1->Get("http://localhost:1985/stream");

if (IdHTTP1->ResponseCode != 200 || !IsHeaderMediaType(IdHTTP1->Response->ContentType, "multipart/x-mixed-replace"))
    return;

String LBoundary = ExtractHeaderSubItem(IdHTTP1->Response->ContentType,"boundary", QuoteHTTP);
// String LBoundary = IdHTTP1->Response->RawHeaders->Params["Content-Type"][ "boundary"];

String LBoundaryStart = _D("--") + LBoundary;
String LBoundaryEnd = LBoundaryStart + _D("--");

unique_ptr<TIdTCPStream> Stream(new TIdTCPStream(IdHTTP1, 0));

do
    {
    String LLine = IdHTTP1->IOHandler->ReadLn();

    if (LLine == LBoundaryStart) break;
    if (LLine == LBoundaryEnd) return;
    }
while (true);

ThreadWait(FuncBind( &TMain::ProcessStreaming, this, Stream.get(), LBoundary));
}
//---------------------------------------------------------------------------

Code:
void TMain::ProcessImage(TMemoryStream *LMStream)
{
Image1->Bitmap->LoadFromStream(LMStream);
}
//---------------------------------------------------------------------------
void TMain::ProcessStreaming(TIdTCPStream* Stream, String LBoundary)
{
IdHTTP1->IOHandler->LargeStream = true;
TIdMessageDecoder* LDecoder = new TIdMessageDecoderMIME(nullptr);
try
    {
    bool LMsgEnd = false;

//    do
//        {
        static_cast<TIdMessageDecoderMIME*>(LDecoder)->MIMEBoundary = LBoundary;

        LDecoder->SourceStream = Stream;
        LDecoder->FreeSourceStream = false;
        LDecoder->ReadHeader();

        switch (LDecoder->PartType)
            {
            case mcptText:
            case mcptAttachment:
                {
                TMemoryStream *LMStream = new TMemoryStream;
                try
                    {
                    TIdMessageDecoder* LNewDecoder = LDecoder->ReadBody(LMStream, LMsgEnd);
                    LMStream->Position = 0;
                    LMStream->SaveToFile("image.png");

                    try
                        {

//                        SynchronizeUI(FuncBind( &TMain::ProcessImage, this, LMStream));
                        }
//                    catch (const Exception &)
//                        {
//                        delete LNewDecoder;
//                        throw;
//                        }
                    __finally
                        {
                        delete LDecoder;
                        LDecoder = LNewDecoder;
                        }

                    }
                __finally
                    {
                    delete LMStream;
                    }
                }
                break;
            case mcptIgnore:
                {
                FreeAndNil(LDecoder);
                LDecoder = new TIdMessageDecoderMIME(nullptr);
                static_cast<TIdMessageDecoderMIME*>(LDecoder)->MIMEBoundary = LBoundary;
                }
                break;
            case mcptEOF:
                {
                FreeAndNil(LDecoder);
                LMsgEnd = true;
                }
                break;
            }

//        } while (LDecoder && !LMsgEnd);
    }
__finally
    {
    delete LDecoder;
    }

You will find a lot of commented code because I was trying some other stuff in order to make it work.
When I save the image stream to a file on the server side before pushing it, it's ok and a valid image png file. But when I do save it the stream on the client side it shows that it is not a valid png file.

As you can see I am only testing with one frame and it is invalid file
Reply
#5
(05-03-2022, 06:51 AM)Ahmed Sayed Wrote: I have an issue with images being sent from server.

One issue I see is you are not synchronizing concurrent access to your TBitmap object (a Sleep() is not good enough). Your HTTPServer's OnCommandGet handler runs in a different thread than your camera's OnSampleBufferReady handler, so you need to coordinate access to the TBitmap, such as with a TCriticalSection or equivalent.

Another issue - since you are sending raw image bytes, try adding a Content-Transfer-Encoding header to each image, set to either 8bit or binary. TIdMessageDecoderMIME in modern Indy shouldn't need this header, but it may be needed for an older Indy, or non-Indy clients.

Code:
IOHandler->WriteLn("Content-Type: image/png");
IOHandler->WriteLn("Content-Transfer-Encoding: binary"); // <-- here
IOHandler->WriteLn();
IOHandler->Write(str.get());
IOHandler->WriteLn();
IOHandler->WriteLn("--imgboundary");

(05-03-2022, 06:51 AM)Ahmed Sayed Wrote: I am using Berlin Upd2 and FMX camera component.

Berlin is 6 years old, are you using an up-to-date Indy, at least?

(05-03-2022, 06:51 AM)Ahmed Sayed Wrote: The data received on client side is bigger than the one sent from the server. I don't know why?

Bigger how, exactly? Can you provide an example?

Also, have you tried connecting a standard web browser to your streaming server? Is the browser able to display the updated images?

(05-03-2022, 06:51 AM)Ahmed Sayed Wrote: When I save the image stream to a file on the server side before pushing it, it's ok and a valid image png file. But when I do save it the stream on the client side it shows that it is not a valid png file.

Then you should be able to compare the raw bytes of the 2 files and see exactly what is different between them. What is the actual difference? The first thing that comes to mind is most likely either $0A bytes being converted into $0D $0A, and/or extra $0D $0A being inserted, into the decoded output. Neither of which should happen if modern TIdMessageDecoderMIME detects that the data stream is carrying binary data.

Reply
#6
No I use Indy version installed in berlin upd2 I did not update to any new version, I am planning to but I am waiting for a more stable version regarding clanc and code completion.

Anyway, I managed to make it work with TCP and TFTP components from Indy. I am only testing right now but actually this might lead me to create something like a video chat/meetings application for our company, sure VoIP is another story but my question is:

I wanted to test the speed of streaming over the internet not just on my PC. I couldn't connect to my own PC using my public IP address. I had to use a tunneling program called ngrok in order to do it. And it was really slow. 

How software like Zoom, MS Teams do these video chats programs to be so fast with good quality? What protocols they use? if there is, Can it be found in Indy framework?

Also, How can I connect to my own machine from the internet from the same machine (I know it is stupid but I only got one PC) without using tunneling? Shouldn't a public IP be enough? Do I have to use special Indy component for it?
Reply
#7
(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: No I use Indy version installed in berlin upd2

Then I strongly suggest you update Berlin to the latest Indy. Like I said, your version is quite old. If the data size issue persists after that, we can dig further into it.

(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: I am planning to but I am waiting for a more stable version regarding clanc and code completion.

That doesn't affect your ability to update your version of Indy installed in Berlin. You don't need the latest Delphi to use the latest Indy.

(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: I wanted to test the speed of streaming over the internet not just on my PC. I couldn't connect to my own PC using my public IP address. I had to use a tunneling program called ngrok in order to do it. And it was really slow. 

You shouldn't need to use tunneling software to reach your PC from the Internet. Simply have Indy open a listening IP/Port on your PC, and then configure your Internet router to forward a corresponding WAN IP/Port to your LAN PC's IP/port. Also, make sure your ISP is not blocking incoming connections from the Internet to your router (some ISPs charge more for that feature).

(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: How software like Zoom, MS Teams do these video chats programs to be so fast with good quality?

They use protocols that are optimized for media transmission. For instance, they use UDP-based (not TCP-based) transports like RTP/H323, etc. And they utilize media compression to reduce packet sizes, etc.

(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: What protocols they use? if there is, Can it be found in Indy framework?

There are some compression components in Indy, at least, but there are no components/protocols implemented for streaming media, no. But you can use Indy components as the basis for your own implementations (though, I recommend using other pre-existing 3rd party libraries that are specialized for streaming media, as these tend to be quite complex protocols to implement from scratch).

(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: Also, How can I connect to my own machine from the internet from the same machine (I know it is stupid but I only got one PC) without using tunneling? Shouldn't a public IP be enough?

Whether or not you can connect to your router's WAN IP/Port from the LAN side depends on your router's internal capabilities. But chances are that connecting directly to your WAN IP/Port from the LAN side will not work as they are physically separated, you typically need an actual remote client running somewhere on the Internet to reach the router's WAN connection. In which case, it makes sense that you would need a tunnel like ngrok to connect to a remote proxy that can then connect back to you via you router's WAN. Otherwise, just give your client software to a friend and have them connect to your public WAN IP/port from their PC.

But, do you really need such a remote connection just to test your code? You can connect a local LAN client to your LAN server using the server's LAN IP, or even 'localhost' if they are on the same PC.

(05-04-2022, 03:51 AM)Ahmed Sayed Wrote: Do I have to use special Indy component for it?

No.

Reply
#8
The reason I want to test this from a remote PC is that because some employees might be using it from home during the Corona Pandemic GOD knows whats coming next?

But I was wondering if used TIdTriviaFTP with stream compression would be faster than using TCP?

Also, How do companies who made Zoom for example manage to connect 2 computers in a video call without me have to configure anything in my router? Do they use tunneling too?
Reply
#9
(05-05-2022, 01:07 AM)Ahmed Sayed Wrote: But I was wondering if used TIdTriviaFTP with stream compression would be faster than using TCP?

You are comparing apples and oranges.

(05-05-2022, 01:07 AM)Ahmed Sayed Wrote: Also, How do companies who made Zoom for example manage to connect 2 computers in a video call without me have to configure anything in my router? Do they use tunneling too?

Have a look at NAT traversal techniques, for starters.

Reply
#10
Ok, I read some about TCP/UDP hole punching and I have seen some people asking about it on stack-overflow but I don't understand how to implement it in Indy? Regardless of the protocol HTTP/TCP/UDP.

I believe the idea is that every computer behind a NAT has the 2 components client/server. Let's say that I already know both PCs public IPs beforehand and the ports we are using (setting on used server component TId*Server).

For example:

PC 1:
Private IP: 192.168.1.3
Public IP: 102.186.14.167 //Router IP
TIdTCPServer on Port: 1985

On another location behind another NAT (Router)
PC 2
Private IP: 192.168.1.55
Public IP: 102.205.36.52 //Router IP
TIdTCPServer on Port: 65478

I assume that these ports used in the apps are local ports and are not public and can't be reach from the public internet that's why both apps will not be able to connect to each other unless I open ports on both routers and map them to the local ports then use these public ports instead. But I can't do that for every user that will use this app.

I have seen this question: https://stackoverflow.com/questions/4155...on-refused

I don't understand the idea behind using 4 sockets on each client? I am not planning on making a TIdTCPClient connect to another TIdTCPClient here, in fact I want this to be easy as possible. I wanna use a normal connection between a client and a server on both sides.

Now, how can I achieve this with Indy TCP for example? How can I get the public port? Knowing that I still don't have a "Rendezvous Server". So, for testing purposes this will be an HTTP tunnel using ngrok and that Rendezvous server will be also on PC1.
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)