RE: OPCUA SSL problem

[Majer Karel via Tech-talk <tech-talk at aps.anl.gov>]

Hi Ralph,

 

Thank you for your answer. I forgot to mention that the client certificate and UAExpert certificate were added into the trusted list on the server.

 

Our setup is actually more complicated, because we run the IOC in a Docker container which has OpenSSL 3. The client certificate was created on host machine, which I just found uses OpenSSL 1.1.1. Application URI and DNS is set up correctly for the IOC using the SubjectAltName field. Hostname of the Docker container and the host machine are the same. The container is started with --network=host option, so it has the same IP as host machine (this is actually necessary for Channel Access to work).

[2024-10-02 12:46:41.331 (UTC+0000)] warn/client        The configured ApplicationURI does not match the URI specified in the certificate for the SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None

Similar message actually appears on startup of every IOC, even the working ones. If a client certificate is not set (we tried that as well), it appears for every security policy, with client certificate just for None.

 

We actually have a reason to believe that the trust issue is on client side. We have recorded working communication with UAExpert and non-working with open62541 with Wireshark. After a Hello message and acknowledgment from the server the client sends OpenSecureChannelRequest with SecurityPolicy#None. Server responds with OpenSecureChannelResponse which contains server certificate. Client certificate has not been sent yet at this point. UAExpert continues with GetEndpointsRequest, but open62541 closes TCP communication (FIN is first sent by the client).

Client certificate is sent by UAExpert at a later point when a new encrypted channel is created. We never get there with open62541.

 

I also tried to use the UAExpert certificate as IOC’s client certificate (it required hacking the IOC source code to replace its URI with UAExpert’s) and got the same result.

 

Regarding the placement of the certificates, I played with UAExpert certificate store and found two options to connect without any trust complaints:

1.      CA certificate in PKI/trusted/certs, CA crl in PKI/trusted/crl

2.      CA certificate in PKI/issuers/certs, CA crl in PKI/issuers/crl, server certificate in PKI/trusted/certs

 

We are getting out of ideas now and starting to suspect the open62541 library, but it’s hard for us to understand at which point and how is the server certificate verified by the client.

 

Kind regards,

 

Karel

 

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Ralph Lange via Tech-talk
Sent: Thursday, October 3, 2024 3:09 PM
To: EPICS Tech Talk <tech-talk at aps.anl.gov>
Subject: Re: OPCUA SSL problem

 

Hi Karel,

 

Disclaimer up front: I am not using the open62541 client (yet) and I'm not using Secure OPC UA. Take my answers with a grain of salt.

 

On Wed, 2 Oct 2024 at 15:16, Majer Karel via Tech-talk <tech-talk at aps.anl.gov> wrote:

 

I am using EPICS OPCUA module built with Open62541 v1.3.10 and OpenSSL, running on Rocky Linux 9. It is mostly used to connect to B&R PLCs which have self-signed server certificate and then it runs without problems.

I need to communicate also with another server which has certificate signed by a certification authority. The authority uses a self-signed certificate. I cannot get this communication working in any setup.

 

Rocky 9 means using OpenSSL v3, correct?

 

This is the output of EPICS shell:

 

[root@fct-deploy20 iocopcua]# ./st.cmd

 

[... all good ...]

 

Session PLC: (connect) setting up PKI provider

Session PLC: (setupIdentity) setting Anonymous token

Session PLC: (setupSecurity) reading endpoints from opc.tcp://192.168.0.250:4840

[2024-10-02 12:46:41.331 (UTC+0000)] warn/client        The configured ApplicationURI does not match the URI specified in the certificate for the SecurityPolicy http://opcfoundation.org/UA/SecurityPolicy#None

 

Haven't seen that before.

Does the Application URI in your client certificate match what the IOC uses when connecting? (The command `opcuaShowSecurity` shows the URI.)

 

[2024-10-02 12:46:41.335 (UTC+0000)] info/server        Reloading the trust-list

[2024-10-02 12:46:41.336 (UTC+0000)] info/server        Reloading the issuer-list

[2024-10-02 12:46:41.336 (UTC+0000)] info/server        Reloading the revocation-list

[2024-10-02 12:46:41.336 (UTC+0000)] warn/channel       Connection 8 | SecureChannel 0 | Receiving the response failed with StatusCode BadCertificateUntrusted

 

This sounds to me like the server didn't trust the client certificate.

Again - no experience with open62541, so I'm not sure about the exact meaning of the error message.

 

[2024-10-02 12:46:41.336 (UTC+0000)] info/client        Client Status: ChannelState: Closed, SessionState: Closed, ConnectStatus: BadConnectionClosed

Session PLC: (setupSecurity) UaDiscovery::getEndpoints from opc.tcp://192.168.0.250:4840 failed with status BadConnectionClosed

OPC UA session PLC: security discovery and setup failed with status cantConnect

 

We used tcpdump to capture TCP communication, it shows TCP is closed by client after receiving OpenSecureChannelResponse from the server.

We have certificate of both the authority and the server and crl of the authority. This is a list of steps which we tried and still got BadCertificateUntrusted:

·         Putting server certificate into PKI/trusted/certs

 

This should work. (Not using the CA signature.)

 

·         Putting CA certificate into PKI/trusted/certs and crl into PKI/trusted/crl

·         Putting CA certificate into PKI/issuers/certs and crl into PKI/issuers/crl

 

Alternatively, this should work, using the CA. (Without the server certificate in PKI/trusted/certs.)

 

·         Combinations of all of the above

·         Adding certificates to the system

 

This should not work.

 

·         Running with sec-mode=None and commenting out opcuaSetupPKI command

 

What about commenting out the client certificate setup?

 

We also tried running the IOC with strace to see system calls and noticed that PKI/trusted/crl directory was not opened during IOC startup.

We have used openssl verify to check the server certificate against the CA with OK result.

 

So the server certificate seems fine.

 

 And perhaps most importantly, we can connect to the server with UAExpert without any issues when we use the same certificates.

 

Yet another pointer that the server certificates are not the problem, but the IOC's client certificate is.

Faults of the client certificate may lead to the server rejecting it.

 

Check the client certificate that UAExpert uses (gets created when you first start the application) against your IOC's certificate.

Does the Application URI (that contains the name of the IOC) match what the IOC is using?

Does the IP address and/or DNS name of the IOC host match? If you use a DNS name: does it resolve forward and backward from/to the IOC host name and address?

Are the certificate properties set up in a way that matches the UAExpert certificate? (In the security README I am describing the minimal set that I found working.)

 

If the server is picky: renaming the IOC, running it on a different host or renaming the host will all require generating a new client CA.

 

Do you have any experience with this behavior and any advice how to fix it or even where to look for the problem?

 

Be patient. Don't get frustrated.

This system is designed to make it hard to get things right. Enjoy cyber security at work.

 

If the secure channel is closed by the server before the IOC can even read the endpoints, I would assume a problem with the client certificate.

 

The best of luck!
Please give feedback when you get it working.

 

Cheers,
~Ralph