Posts: 7
Threads: 1
Joined: Dec 2018
Reputation:
0
Hello!
I found an issue by accident in TIdSSLIOHandlerSocketOpenSSL.OnStatusInfoEx for the need of writing an NSS-file to decrypt TLS-1.2-communication with Wireshark. It seems that the declaration of SSL3_STATE in IdSSLOpenSSLHeaders.pas shifts the memory content from the DLL one byte to left. When i shift the S3-Pointer via Dec(PByte(S3)) also one byte to left so it will work fine.
Please have a look for this thread in the german Delphi Praxis forum for sample codes and screen captures from IDE debug.
Greetz
Cody
Posts: 649
Threads: 2
Joined: Mar 2018
Reputation:
35
Location: California, USA
12-04-2018, 07:38 PM
(This post was last modified: 12-04-2018, 07:41 PM by rlebeau.)
(12-04-2018, 10:05 AM)Codehunter Wrote: I found an issue by accident in TIdSSLIOHandlerSocketOpenSSL.OnStatusInfoEx for the need of writing an NSS-file to decrypt TLS-1.2-communication with Wireshark. It seems that the declaration of SSL3_STATE in IdSSLOpenSSLHeaders.pas shifts the memory content from the DLL one byte to left. When i shift the S3-Pointer via Dec(PByte(S3)) also one byte to left so it will work fine.
I have checked in a fix for IdSSLOpenSSLHeaders.pas. There was indeed a 1-off error in the declaration of the SSL3_STATE.write_mac_secret field, which is right above the server_random and client_random fields.
That being said, you really should not be accessing OpenSSL's record fields directly anymore. Use appropriate accessor functions from the OpenSSL API instead. In fact, once Indy is (eventually) updated to support OpenSSL 1.1+, direct access to record fields will no longer be possible at all, as ALL of OpenSSL's records were changed to opaque types in 1.1, so you MUST use accessor functions only. So, you may as well get in the habit of doing it now, because it is coming down the line in the future.
Posts: 7
Threads: 1
Joined: Dec 2018
Reputation:
0
(12-04-2018, 07:38 PM)rlebeau Wrote: I have checked in a fix for IdSSLOpenSSLHeaders.pas. There was indeed a 1-off error in the declaration of the SSL3_STATE.write_mac_secret field, which is right above the server_random and client_random fields.
That being said, you really should not be accessing OpenSSL's record fields directly anymore. Use appropriate accessor functions from the OpenSSL API instead. In fact, once Indy is (eventually) updated to support OpenSSL 1.1+, direct access to record fields will no longer be possible at all, as ALL of OpenSSL's records were changed to opaque types in 1.1, so you MUST use accessor functions only. So, you may as well get in the habit of doing it now, because it is coming down the line in the future.
Thank you for the quick fix! Prior i had read some discussions about Indy and TLS 1.3 including your statements about OpenSSL 1.1.1. I was thinking that these major changes in OpenSSL API would lead to heavy work on Indy for you and your coworkers. But i had not found a final decision from you, wheather Indy will support OpenSSL 1.1.1 in the future or not. Only we have a choice? The day will come and TLS 1.3 is state-of-the-art. Often, one of the big players (mostly Mozilla or Google) decide to drop the support of older encryption from their browsers. Not much later and many servers drops offering older encryption. Some of the REST servers for which i use Indy offers only TLS 1.2 anymore. When OpenSSL removes some crappy stuff from their API, i think this could be beneficial to Indy in the future.
When i should use OpenSSL directly, i need a re-design of definition how many hours a day have. 24 are not enough to me
Posts: 649
Threads: 2
Joined: Mar 2018
Reputation:
35
Location: California, USA
12-05-2018, 07:02 PM
(This post was last modified: 12-05-2018, 07:05 PM by rlebeau.)
(12-05-2018, 10:13 AM)Codehunter Wrote: Prior i had read some discussions about Indy and TLS 1.3 including your statements about OpenSSL 1.1.1. I was thinking that these major changes in OpenSSL API would lead to heavy work on Indy for you and your coworkers.
Yes, it will.
(12-05-2018, 10:13 AM)Codehunter Wrote: But i had not found a final decision from you, wheather Indy will support OpenSSL 1.1.1 in the future or not.
We are still discussing the particulars about it, and so there is no ETA at this time. It WILL happen, I just don't know WHEN.
(12-05-2018, 10:13 AM)Codehunter Wrote: The day will come and TLS 1.3 is state-of-the-art.
Yes, but not for a couple of years. It is still very new, not many sites are using it yet. TLS 1.1 and 1.2 will suffice for awhile, so there is still time.
(12-05-2018, 10:13 AM)Codehunter Wrote: Often, one of the big players (mostly Mozilla or Google) decide to drop the support of older encryption from their browsers.
Yes, but TLS 1.1 and 1.2 are not going to be dropped anytime soon (PCI didn't require migrating from TLS 1.0 to 1.1 until just earlier this year). The big players, and many websites, have only just recently started dropping TLS 1.0, and that has been around a long time (since 1999). For instance, Apple, Microsoft, Google, and Mozilla are not going to drop TLS 1.0 and 1.1 until 2020.
(12-05-2018, 10:13 AM)Codehunter Wrote: Some of the REST servers for which i use Indy offers only TLS 1.2 anymore.
Which is fine, since Indy supports TLS 1.2 today ( it is just not enabled by default).
Posts: 7
Threads: 1
Joined: Dec 2018
Reputation:
0
(12-05-2018, 07:02 PM)rlebeau Wrote: Yes, but not for a couple of years. It is still very new, not many sites are using it yet. TLS 1.1 and 1.2 will suffice for awhile, so there is still time.
...
Which is fine, since Indy supports TLS 1.2 today (it is just not enabled by default).
I agree with you. The cause WHY i insist to support TLS 1.3 is a lesson from another front: Since Embarcadero has dropped using Indy in their REST client and switched to native-OS-connections, there are many problems on Windows 7 and Vista machines when connecting to TLS-1.2-only-servers. The support of 1.2 depends from some OS updates and registry patches. Much work for our field representatives at the customers side. Since we decide to drop using Delphi REST client and built our own one with Indy+OpenSSL, there are NO problems anymore with 1.2. In my mind's eye I saw the same problems with Indy and TLS 1.3 coming up.
Posts: 7
Threads: 1
Joined: Dec 2018
Reputation:
0
In conjunction with this, i have a feature request: Since it is very hard to dive into the deep of TLS processing (for non Remy's ^^), it would be very nice to have a easy-to-use way to write standard NSS files. My TIdSSLIOHandlerSocketOpenSSL.OnStatusInfoEx handler (linked in the first post in this thread) is working but a little bit freaky. OnStatusInfoEx isn't the right place to do this because it is fired too often (performance issues)
For example, a TIdSSLIOHandlerSocketOpenSSL.NSSFilePath string property and a TIdSSLIOHandlerSocketOpenSSL.NSSWriteEnabled boolean property.
See https://developer.mozilla.org/en-US/docs...Log_Format for details. Wireshark supports these files for TLS/SSL decryption. It is important to open this file (stream) for shared read-write, because Wireshark can't open it if another program has it open exclusively.
Alternatively, an event handler which is fired once after TLS/SSL connection is finally established, with all neccessary infos (secrets... ) to write a NSS file.
For clarification: I want this only for debugging purposes in the lab. Firefox and Chrome have such a thing, enabled via some developer options.
Thanks!
Posts: 649
Threads: 2
Joined: Mar 2018
Reputation:
35
Location: California, USA
02-18-2019, 07:16 PM
(This post was last modified: 02-18-2019, 07:16 PM by rlebeau.)
(02-16-2019, 09:50 PM)Codehunter Wrote: it would be very nice to have a easy-to-use way to write standard NSS files.
That is outside of Indy's scope. I'm sure you can find a 3rd party that has already done this. Doesn't Mozilla's own NSS library have something you can use for this?
Doing some research, it seems like it would not be very difficult for you to write a custom function in your own code:
How do I extract the pre-master secret using an OpenSSL-based client?
Extract pre-master keys from an OpenSSL application
(02-16-2019, 09:50 PM)Codehunter Wrote: My TIdSSLIOHandlerSocketOpenSSL.OnStatusInfoEx handler (linked in the first post in this thread) is working but a little bit freaky. OnStatusInfoEx isn't the right place to do this because it is fired too often (performance issues)
OnStatusInfo/Ex is fired at different stages of the connection, including the start and end of a handshake. You are not trying to write your file on every stage, are you?
(02-16-2019, 09:50 PM)Codehunter Wrote: For example, a TIdSSLIOHandlerSocketOpenSSL.NSSFilePath string property and a TIdSSLIOHandlerSocketOpenSSL.NSSWriteEnabled boolean property.
I'll consider it, but don't rely on it being added any time soon.
(02-16-2019, 09:50 PM)Codehunter Wrote: Alternatively, an event handler which is fired once after TLS/SSL connection is finally established, with all neccessary infos (secrets... ) to write a NSS file.
What is stopping you from doing that today? OnStatusInfoEx gives you notification when the handshake is finished (the AWhere parameter is SSL_CB_HANDSHAKE_DONE), whether it was successful or not (the ARet parameter), and direct access to OpenSSL's SSL/TLS session object (the ASslSocket parameter).
(02-16-2019, 09:50 PM)Codehunter Wrote: For clarification: I want this only for debugging purposes in the lab. Firefox and Chrome have such a thing, enabled via some developer options.
Firefox and Chrome use there own built-in SSL/TLS engines, so of course they would be able to provide debugging options to output their own secret data.
Posts: 7
Threads: 1
Joined: Dec 2018
Reputation:
0
02-20-2019, 07:39 PM
(This post was last modified: 02-20-2019, 07:49 PM by Codehunter.)
(02-18-2019, 07:16 PM)rlebeau Wrote: OnStatusInfo/Ex is fired at different stages of the connection, including the start and end of a handshake. You are not trying to write your file on every stage, are you?
What is stopping you from doing that today? OnStatusInfoEx gives you notification when the handshake is finished (the AWhere parameter is SSL_CB_HANDSHAKE_DONE), whether it was successful or not (the ARet parameter), and direct access to OpenSSL's SSL/TLS session object (the ASslSocket parameter).
This is what I'm tried to do here. I have seen that OnStatusInfoEx is multiple fired and the neccessary arrays are not filled at every stage. AWhere=SSL_CB_HANDSHAKE_DONE is a good hint, this was the detail what I've been missing. As I said before, I'm not a SSL expert. Currently I'm checking every time if the arrays are filled and if so, my NSS file is ready to write.
But: You said that OnStatusInfoEx is not a good idea to do this because for future changes related to OpenSSL 1.1 and TLS 1.3. This is what stopped me today :-) (The prior issue which stopped me was the byte offset) Therefore please understand me right: A request to have a similar event like OnStatusInfoEx in "Indy 11" / future development.
Posts: 649
Threads: 2
Joined: Mar 2018
Reputation:
35
Location: California, USA
02-20-2019, 08:20 PM
(This post was last modified: 02-20-2019, 08:30 PM by rlebeau.)
(02-20-2019, 07:39 PM)Codehunter Wrote: This is what I'm tried to do here.
Yes, you pointed that out earlier in this same discussion thread.
(02-20-2019, 07:39 PM)Codehunter Wrote: I have seen that OnStatusInfoEx is multiple fired and the neccessary arrays are not filled at every stage. AWhere=SSL_CB_HANDSHAKE_DONE is a good hint, this was the detail what I've been missing. As I said before, I'm not a SSL expert.
That is what documentation is good for. The OnStatusInfo/Ex events are triggered by an info callback registered via SSL_CTX_set_info_callback(), it describes what the status parameters mean.
(02-20-2019, 07:39 PM)Codehunter Wrote: Currently I'm checking every time if the arrays are filled and if so, my NSS file is ready to write.
Well, now you can be more strategic about it so you do it at the correct time.
(02-20-2019, 07:39 PM)Codehunter Wrote: But: You said that OnStatusInfoEx is not a good idea to do this because for future changes related to OpenSSL 1.1 and TLS 1.3. This is what stopped me today :-)
That is not what I said.
I never said you would have to stop using the OnStatusInfo/Ex events in OpenSSL 1.1+. Far from it. The events will continue to work the same as before.
Your OnStatusInfo/Ex code is directly accessing internal data fields of OpenSSL's SSL object. So, what I actually said is that you won't be able to directly access the data fields anymore in OpenSSL 1.1+. They have been hidden in OpenSSL 1.1+'s API going forward, tucked behind a new opaque abstraction layer so OpenSSL devs can make future changes to internal structures without worrying about breaking any user code, as was the case prior to OpenSSL 1.1. As such, you will have to use new getter functions that OpenSSL provides (specifically, SSL_SESSION_get_master_key() and SSL_get_client_random() in your situation).
I never said you could not access OpenSSL 1.1+'s internal data at all, only that you will have to change HOW you access the data.
Posts: 7
Threads: 1
Joined: Dec 2018
Reputation:
0
02-21-2019, 05:42 PM
(This post was last modified: 02-21-2019, 05:53 PM by Codehunter.)
Ah, ok then I've misunderstand you. Sorry. :-) But your hint to SSL_CB_HANDSHAKE_DONE is very useful to me. Is the ASslSocket in OnStatusInfoEx suitable to SSL_get_client_random() ?
One last question: Referring to the NSS documentation "CLIENT_RANDOM" is the only valid NSS label at the moment because "RSA" is outdated and all other labels are for TLS 1.3 which is currently not supported by Indy 10. Right?
|