IdTCPClient on Android: Connecting to non-active network

[rlebeau]

But now I get an AV:
...
TCPThread.Execute: Start main loop
ERROR: TCPThread.Execute: GStack failed: WIFIInterface=wlan0 Length(WIFIInterface)=5: Access violation at address 90CC4666, accessing address 00000014

An AV at a memory address near 0 usually means you are accessing a nil pointer.  My guess would be the TIdTCPClient.Socket.Binding object hasn't been created yet when you call setsockopt().  Try moving that call into the TIdTCPClient.OnSocketAllocated or TIdTCPClient.OnAfterBind event instead.  That ensures the Socket.Binding object and its socket Handle are both ready for use.

[BartKindt]

An AV at a memory address near 0 usually means you are accessing a nil pointer.  My guess would be the TIdTCPClient's Socket.Binding object hasn't been created yet when you call setsockopt().  Try moving that call into the TIdTCPClient's OnSocketAllocated or OnAfterBind event instead.  That ensures the Socket.Binding object and its socket Handle are both ready for use.

Okay. Fixed the AV. Now I have a socket error:

Code:

procedure TTCPThread.IdTCPClient1OnAfterBind(ASender: TObject);
var
 M: TMarshaller;
begin
 LocalLog('IdTCPClient: OnAfterBind: ');
 if NetworkData.LocalIP <> '' then
 begin
   try
     if NetworkData.WIFIInterface <> '' then
     begin
       if NOT assigned(GStack) then // just checking...
       begin
         LocalLog('IdTCPClient: GStack is not assigned!',d_error);
         exit;
       end;
       LocalLog('IdTCPClient1.Socket.Binding.Handle='+IntToStr(IdTCPClient1.Socket.Binding.Handle)); // Result: "42"
       GStack.CheckForSocketError(
         Posix.SysSocket.setsockopt(
         IdTCPClient1.Socket.Binding.Handle,
         Id_SOL_SOCKET,
         SO_BINDTODEVICE,
         M.AsAnsi(NetworkData.WIFIInterface).ToPointer^,
         Length(NetworkData.WIFIInterface)+1 // assuming the device name uses ASCII chars only
         )
       );
     end;
   except
     on E:Exception do LocalLog('TCPThread: GStack failed: WIFIInterface='+NetworkData.WIFIInterface
       +' Length(WIFIInterface)='+IntToStr(length(NetworkData.WIFIInterface))
       +': '+E.Message,d_error);
   end;
 end; // ELSE: We are now using the Android Active Network, whatever that may be.

end;

This generates:

IdTCPClient1.Socket.Binding.Handle=42
ERROR: TCPThread: GStack failed: WIFIInterface=wlan0 Length(WIFIInterface)=5: Socket Error # 1

I do hope we are not in a situation where Android somehow blocks any attempt to connect to a 'non-active network'.
But if this is at Linux level, how could that be?
So, the WIFI IS connected (enabled) and the name 'wlan0' is valid, as I can see a lot of data passing through the Android debug window

PS, this is what the unfiltered Android Log looks like. There is no other issue reported by Android during the call with GStack.
You can see the WIFI is connected to my test WIFI (with no Internet) called "SARTrack_SAT" on interface "wlan0"

Code:

45: I/info(16249): SARTrack Service: [S] D3: IdTCPClient: OnSocketAllocated:
05-09 15:28:17.547: I/info(16249): SARTrack Service: [S] D3: IdTCPClient: OnAfterBind:
05-09 15:28:17.549: I/info(16249): SARTrack Service: [S] D3: IdTCPClient1.Socket.Binding.Handle=46
05-09 15:28:17.551: E/error(16249): SARTrack Service: ERROR: [S] ERROR: TCPThread: GStack failed: WIFIInterface=wlan0 Length(WIFIInterface)=5: Socket Error # 1
05-09 15:28:17.552: I/info(16249): SARTrack Service: [S] D3: SSL Status: 1  Connecting to 192.168.53.104.
05-09 15:28:18.131: D/WifiStateMachine(1613):  ConnectedState !CMD_RSSI_POLL  rt=10152611/10711287 18 0 "SARTrack_SAT" c8:3a:35:15:10:28 rssi=-39 f=2437 sc=60 link=11 tx=0.2, 0.0, 0.0  rx=0.7 bcn=0 [on:0 tx:0 rx:0 period:3000] from screen [on:0 period:1159108051] hn rssi=-34 ag=0 hr ls-=0 [56,56,56,56,61] brc=0 lrc=0 offload-stopped
05-09 15:28:18.132: D/WifiStateMachine(1613):  L2ConnectedState !CMD_RSSI_POLL  rt=10152612/10711288 18 0 "SARTrack_SAT" c8:3a:35:15:10:28 rssi=-39 f=2437 sc=60 link=11 tx=0.2, 0.0, 0.0  rx=0.7 bcn=0 [on:0 tx:0 rx:0 period:1] from screen [on:0 period:1159108052] hn rssi=-34 ag=0 hr ls-=0 [56,56,56,56,61] brc=0 lrc=0 offload-stopped
05-09 15:28:18.133: D/WifiStateMachine(1613):  get link layer stats 0
05-09 15:28:18.133: D/WifiNative-wlan0(1613): doString: [SIGNAL_POLL]
05-09 15:28:18.134: D/wpa_supplicant(1852): wlan0: Control interface command 'SIGNAL_POLL'
05-09 15:28:18.151: D/WifiStateMachine(1613): fetchRssiLinkSpeedAndFrequencyNative rssi=-40 linkspeed=11 freq=2437
05-09 15:28:18.151: D/WifiConfigManager(1613): updateConfiguration freq=2437 BSSID=c8:3a:35:15:10:28 RSSI=-39 "SARTrack_SAT"WPA_PSK
05-09 15:28:18.151: D/WifiStateMachine(1613): calculateWifiScore freq=2437 speed=11 score=60 highRSSI  -> txbadrate=0.00 txgoodrate=0.09 txretriesrate=0.00 rxrate=0.34 userTriggerdPenalty0
05-09 15:28:18.151: D/WifiStateMachine(1613):  good link -> stuck count =0
05-09 15:28:18.151: D/WifiStateMachine(1613):  badRSSI count0 lowRSSI count0 --> score 56
05-09 15:28:18.151: D/WifiStateMachine(1613):  isHighRSSI       ---> score=61
05-09 15:28:18.153: I/QCNEJ(2313): |CORE| CNE received action RSSI/Link Changed events: android.net.wifi.RSSI_CHANGED
05-09 15:28:18.153: D/QCNEJ(2313): |CORE| Updating RSSI: -40
05-09 15:28:18.153: I/QCNEJ(2313): |PB_MSG| sendWifiStatus - subType: 101 networkState: 1 wifiState: 3 rssi=-40 freqBand = _2GHz ssid=SARTrack_SAT bssid=c8:3a:35:15:10:28 ipV4Addr=192.168.53.100 ifNameV4=wlan0 ipAddrV6= ifNameV6= timeStamp:2018-05-09 15:28:18.153 net handle=446693034718 isAndroidValidated = false DNS addrs= 212.56.224.40, 212.56.224.41, 0.0.0.0, 0.0.0.0,
05-09 15:28:21.153: D/WifiStateMachine(1613):  ConnectedState !CMD_RSSI_POLL  rt=10155633/10714309 18 0 "SARTrack_SAT" c8:3a:35:15:10:28 rssi=-40 f=2437 sc=60 link=11 tx=0.1, 0.0, 0.0  rx=0.3 bcn=0 [on:0 tx:0 rx:0 period:3000] from screen [on:0 period:1159111073] hn rssi=-35 ag=0 hr ls-=0 [56,56,56,56,61] brc=0 lrc=0 offload-stopped
05-09 15:28:2

Bart

[rlebeau]

Code:

if NOT assigned(GStack) then // just checking...

You don't need that check. The GStack pointer is always valid whenever an Indy component is alive.

Code:

Posix.SysSocket.setsockopt(
         IdTCPClient1.Socket.Binding.Handle,
         Id_SOL_SOCKET,
         SO_BINDTODEVICE,
         M.AsAnsi(NetworkData.WIFIInterface).ToPointer^,
         Length(NetworkData.WIFIInterface)+1 // assuming the device name uses ASCII chars only
         )

Not sure if this applies to Android's flavor of Linux, but on some Linux platforms, SO_BINDTODEVICE takes an ifreq struct as input instead of a null-terminated string pointer, eg:

Code:

struct ifreq ifr;
memset(&ifr, 0, sizeof(struct ifreq));
snprintf(ifr.ifr_name, sizeof(ifr.ifr_name), "eth0");
ioctl(fd, SIOCGIFINDEX, &ifr);
setsockopt(fd, SOL_SOCKET, SO_BINDTODEVICE,  (void*)&ifr, sizeof(struct ifreq));

Which would translate to Delphi as (roughly):

Code:

uses
  ... Posix.NetIf, Posix.StrOpts, Posix.SysSocket;

const
  SIOCGIFINDEX = $8933;

var
  ifr: Posix.NetIf.ifreq;

FillChar(ifr, Sizeof(ifr), 0);
StrLCopy(ifr.ifr_name, M.AsAnsi(NetworkData.WIFIInterface).ToPointer, IFNAMSIZ-1);
Posix.StrOpts.ioctl(IdTCPClient1.Socket.Binding.Handle, SIOCGIFINDEX, @ifr);
Posix.SysSocket.setsockopt(IdTCPClient1.Socket.Binding.Handle, Id_SOL_SOCKET, SO_BINDTODEVICE, ifr, SizeOf(ifr))

ERROR: TCPThread: GStack failed: WIFIInterface=wlan0 Length(WIFIInterface)=5: Socket Error # 1

Error #1 is EPERM ("Operation not permitted"). You likely need root/sudo access to your Android device to bind to a specific interface by name.

[BartKindt]

I am getting quite desperate.
I managed to get your new code to work, but I still cannot get connected to the WIFI network, when the Android 'Active Network' is set to MOBILE.

Code:

uses Posix.SysSocket, Posix.NetIf, Posix.Termios, Posix.StrOpts

function StrToIfreq(Value:string):Posix.NetIf.ifreq;
var
 i: integer;
 TestStr: RawByteString;
begin
 FillChar(Result, Sizeof(Result), 0);
 TestStr := 'wlan1'; // <<<<< When using a non-existing name, the ioctl gives a "-1" error result. Else it gives a "0"
 System.SetCodePage(TestStr,28591,false);
 for i := 0 to length(TestStr) do
 begin
   Result.ifrn_name[i] := ord(TestStr[i]);
 end;
end;


procedure TTCPThread.IdTCPClient1OnAfterBind(ASender: TObject);
const
 SIOCGIFINDEX = $8933;
var
 M: TMarshaller;
 ifr: Posix.NetIf.ifreq;
 AResult: integer;
begin
 LocalLog('IdTCPClient: OnAfterBind: ');
 if NetworkData.LocalIP <> '' then
 begin
   try
     if NetworkData.WIFIInterface <> '' then
     begin
       LocalLog('IdTCPClient1.Socket.Binding.Handle='+IntToStr(IdTCPClient1.Socket.Binding.Handle)); // Result: "46"

       ifr := StrToIfreq(NetworkData.WIFIInterface);
       AResult := ioctl(IdTCPClient1.Socket.Binding.Handle, SIOCGIFINDEX, @ifr);
       LocalLog('ioctl returned: '+IntToStr(AResult));
       Posix.SysSocket.setsockopt(IdTCPClient1.Socket.Binding.Handle, Id_SOL_SOCKET, SO_BINDTODEVICE, ifr, SizeOf(ifr))

     end;
   except
     on E:Exception do LocalLog('TCPThread: GStack failed: WIFIInterface='+NetworkData.WIFIInterface
       +' Length(WIFIInterface)='+IntToStr(length(NetworkData.WIFIInterface))
       +': '+E.Message,d_error);
   end;
 end; // ELSE: We are now using the Android Active Network, whatever that may be.

end;

What happens is this.
1) When Android sets the Active Network to 'wlan0', and I use same name in the ioctl, this function gives a result of '0'. AND I now get connected to the WIFI based server.
2) When Android sets its Active Network to the MOBILE network, the ioctl return still '0', but I do NOT get any connection to local WIFI server, nothing goes out over the WIFI network.
3) Using situation (1), but now I use a non-valid interface of 'wlan1', the ioctl gives an error result '-1' as it should. But then afterwards, I still get connected to the WIFI server: that is, after the failure of the ioctl, it reverts back to the 'default' network of 'wlan0' (which is active at that moment).
This means that the ioctl works correctly. But I still do net get access to the 'wlan0' interface, unless Android has set this to the "Active Network".
And note that the ioctl return a successful '0'. I am not sure however, if there is another error hidden somewhere in the interface
What else can I do?
Can you please explain to me how Indy knows to which of the two networks to connect all by itself?
There are TWO working interfaces available.
But Indy automatically select the one which Android has set to the 'Active Network'.
How does this work then? As Indy works directly with the Linux interfaces?
Why would Indy use the one Android selects at Linux level? E.g. What makes it a "default"??
Thanks, Bart

[rlebeau]

I managed to get your new code to work, but I still cannot get connected to the WIFI network, when the Android 'Active Network' is set to MOBILE.

I don't know what else to tell you then. Maybe Android is just not physically connected to the WiFi when it is connected to the Mobile network? I'm not an Android expert. There are only so many options available to tell the low-level POSIX socket API which network to use, and we have pretty much exhausted them at this point.

The only thing I have left to suggest is to not use Indy at all, but use Android's own Socket API instead. The Network.bindSocket() method can bind a Socket to a specific network. Or, you can create a Socket on a specific network using the Network.getSocketFactory() and SocketFactory.createSocket() methods.

Can you please explain to me how Indy knows to which of the two networks to connect all by itself?

Indy doesn't pick either one. If you don't explicitly bind the socket to a specific local IP or device name before the socket connection is attempted, the OS picks the appropriate network while connecting the socket to the target host/IP. It picks that network based on available network routes.

But Indy automatically select the one which Android has set to the 'Active Network'.

Indy is not the one making that decision, Android/Linux itself is.

[BartKindt]

Okay Remy,

I was hoping I could bypass the Android system and use Indy's way of selecting the interface.
It looks like it is not possible.

I will try the Android-based ideas you gave me and see what I can and cannot do with that.
The biggest problem is that I cannot find anywhere a forum where I can actually ask Delphi-Android based questions, and get actually a real answer.

I think I have been too spoiled over the many years with the incredible amount of help I got with my Windows based development, and mostly from you...

Many thanks again for all your help,

Bart

[rlebeau]

I was hoping I could bypass the Android system and use Indy's way of selecting the interface.

That would be nice.  But Android is a managed platform, Google doesn't really want users going down to the Linux level unless they really need to.  But, since Delphi Android apps operate inside the Java native NDK, Indy operates at the Linux POSIX level, not at the Android Java level.

I will try the Android-based ideas you gave me and see what I can and cannot do with that.

See Connecting your App to a Wi-Fi Device on the Android Developers Blog.

However, despite what is documented, I see from Android's source code that:

• Network.bindSocket() ultimately calls the undocumented NetworkUtils.bindSocketToNetwork() method, which is implemented in native code, not in Java code.
  
  
• bindSocketToNetwork() calls setNetworkForSocket()
  
  
• setNetworkForSocket() calls FwmarkClient().send() with a SELECT_NETWORK command.
  
  
• FWMARK is an external AF_UNIX socket server running locally on the Android device. When it receives a SELECT_NETWORK command, it calls setsockopt(SOL_SOCKET, SO_MARK) on the original Java Socket descriptor, specifying the network ID number that belongs to the Java Network class.  That ID number is passed by Network.bindSocket() to NetworkUtils.bindSocketToNetwork() (it might be the same ID number returned by WifiConfiguration.networkId when connected to a WiFi network, I'm not sure).
  

And according to Linux's documentation:

SO_MARK (since Linux 2.6.25)
             Set the mark for each packet sent through this socket (similar
             to the netfilter MARK target but socket-based).  Changing the
             mark can be used for mark-based routing without netfilter or
             for packet filtering.  Setting this option requires the
             CAP_NET_ADMIN capability.

So again, you might need root/sudo access to bind a socket to a specific network.  Unless Android's INTERNET or NET_ADMIN permissions enable CAP_NET_ADMIN, but I don't think they do.  So, I'm still at a loss of how to proceed here.  You could try what the blog says, or you could try using SO_MARK directly, and hope it somehow works out OK in your app without messing around with permissions too much.

The biggest problem is that I cannot find anywhere a forum where I can actually ask Delphi-Android based questions, and get actually a real answer.

In the absence of the Embarcadero forums, you can ask Delphi-related questions in the Delphi section of this same forum server, or in the Delphi section of StackOverflow, or any number of other Delphi forums scattered online (though I only frequent the ones I have mentioned).

[BartKindt]

Thanks Remy, I will work my way through the options you mentioned.

Bart