FTP Server support

Support for FTP and FTPS in available in the Neuron® from build 2025-03-20. FTP is an old Internet technology, but with a lot of support in different software and libraries. By providing support for FTP it is now possible to:

Provide broker account holders with File Storage using FTP.
Provide developers and operators with a mechanism to copy, edit and manage files in a simpler way compared to using terminal prompts (chat admin), web pages or remote desktop connections.
Ensure files have consistent access rights in the operating system of the Neuron®. When using Remote Desktop or similar technologies to access the Neuron® for the purpose of copying files to the Neuron®, these might be copied using different access privileges than other types of content files. This might cause problems, such as inability to access the files from the Neuron®, or inability to update them via packages. Files copied to the Neuron® via the FTP interface will use the Neuron’s access rights (Local Service by default) when copying the files, ensuring that the Neuron® will be able to update the files in the future, as part of normal package distributions.

Protocol reference

The FTP protocol is mainly defined in a set of RFC published by the IETF. There are also some custom commands that commonly available, but not standardized. The main RFCs implemented are:

There are some limitations in the implementations (see more details, below):

All FTP access requires authentication (i.e. login, or use of mTLS)
All authentication and transfer require TLS encryption.
Only broker accounts (i.e. XMPP accounts, not administrative user accounts) with FTP access rights explicitly set have access to FTP.
Passive mode is limited to pre-defined ports.

Types of Channels

FTP was created in a time when establishing socket connections was cheaper than implementing streaming or embedding files over a single socket. For this reason, FTP, even for simple tasks requires two or more socket connections between a client (the uploader, downloader or manager of files) and a server (the file server). One connection is made for commands\ - the control channel. All file transfers or lengthy replies (such as a directory listing) are sent over a separate socket - one separate socket per transfer - the data channels. The server needs to manage available ports and provide them, as they are needed, to the different clients.

In the first versions of FTP, the server connected back to the client for each transfer, i.e. each data channel. But as the Internet evolved, and Firewalls were introduced, it became apparent that such an infrastructure is not particularly useful (in spite of NAT). A new type of data channel was introduced: The passive data channel (where the first was considered active, from the server’s point of view). A passive data channel is created by the client connecting to the FTP server using a second connection, establishing a new data connection, on which the transmission of the lengthy response or the file can occur.

TCP/IP Ports

FTP traditionally uses TCP/IP port 21. This port starts unencrypted, but the Neuron® requires the client to switch to TLS before it can authenticate itself. This process is called explicit encryption in some clients. Apart from explicit encryption, there is also an implicit encryption that can be used, often referred to as FTPS (not to be confused with SFTP, which is something else). The default port for FTPS is 990. FTPS works much like HTTPS, in that TLS is used directly upon connecting, and not negotiated through the protocol.

The recommended method to transfer data is through passive data channels. The original definition of passive channels allows for the opening of any port for listening on the server. This is not a good approach, from a security point-of-view, as a very large range of ports must be opened in the firewall for this to work. The Neuron® insteads allows for a limited set of passive data channel ports to be defined (by default 50000 - 50009). The Neuron® then alternately provides access to these ports to the FTP clients that are connected. The Neuron® also ensures the correct client accesses the correct ports. It also ensures to delay access to these passive ports, in case all are used.

The ports used, must be defined defined in the Gateway.config file on the Neuron®. You can change, add or remove ports as you see fit. Every change to the gateway.config file however, requires the Neuron® to be restarted. Example of the Ports section of a Gateway.config file:

<Ports>
	<Port protocol="HTTP">80</Port>
	<Port protocol="HTTPS">443</Port>
	<Port protocol="XMPP.C2S">5222</Port>
	<Port protocol="XMPP.S2S">5269</Port>
	<Port protocol="SOCKS5">1080</Port>
	<Port protocol="SMTP">25</Port>
	<Port protocol="SMTP">587</Port>
	<Port protocol="FTP">21</Port>
	<Port protocol="FTPS">990</Port>
	<Port protocol="FTP.PassiveData">50000</Port>
	<Port protocol="FTP.PassiveData">50001</Port>
	<Port protocol="FTP.PassiveData">50002</Port>
	<Port protocol="FTP.PassiveData">50003</Port>
	<Port protocol="FTP.PassiveData">50004</Port>
	<Port protocol="FTP.PassiveData">50005</Port>
	<Port protocol="FTP.PassiveData">50006</Port>
	<Port protocol="FTP.PassiveData">50007</Port>
	<Port protocol="FTP.PassiveData">50008</Port>
	<Port protocol="FTP.PassiveData">50009</Port>
</Ports>

You can also edit the ports from the Admin menu, under Sources & Nodes:

Note that the ports you define here must also be opened in the Firewall, for incoming TCP/IP connections to be permitted. If you use Azure to host your Neuron®, a firewall entry might look as follows:

Encryption

TLS encryption is mandatory for accessing the Neuron® using FTP. The Neuron® supports both implicit and explicit encryption. The Neuron® also supports Mutual TLS. This means you need to enable Client Certificates for the corresponding port in Gateway.config, and use a valid certificate on the client side when connecting to the Neuron®. If there is an account with FTP access rights having an account name corresponding to the Common Name in the subject field of a valid certificate, there is no need to use a password to access the account. The client will be authenticated as soon as it has provided the user name to the Neuron® FTP server.

Security Note: The weakest link in FTP is the matching of Control Channel with corresponding Data Channels. In active mode (not suitable for Internet use due to firewalls), this problem is not as big, since a third party cannot attempt to hijack a data channel. In passive mode, the client connects to the server twice, first for the control channel where it sends the commands, and secondly for the data channel, where a file transfer will occur. Hypothetically, a third party can hijack this data channel and receive a file not aimed for it. The controls to limit this are basic: First, the data channel is only open for listening exactly after the command has been issued, and only for as long as to receive one connection. This typically generates a window counted in milliseconds. Secondly, the server matches the Remote IP Address of both connections, and closes any data channel connection attempt from another IP address. It is therefore very unlikely that the data channel can be hijacked, but it is possible. To protect against this, it is recommended FTP conections be made using Mutual TLS (mTLS). When mTLS is used, the server also ensures the client certificate used is valid in both connections, and use the same public key. With this added level of protection, FTP becomes very secure.

Account Access Privileges

To give a broker account access rights on a Neuron®, you need to edit the account from the administrative page. At the bottom of the account, you need to provide a Root Folder for the account. This root folder does not have to be unique for the account, but it must exist. You also define whether the account as no access rights (default), read access rights, write access rights or both read and write access rights. You can also provide an optional maximum storage, counted in number of bytes. If you don’t provide a value for the maximum storage, the client will have unlimited access (i.e. until storage runs out). Once the account has been updated, the client can use FTP for file storage, within the folder provided in the configuration (and any of its subfolders). Write-access rights gives the account the right to create new subfolders recursively within the folder provided.

Using an FTP Client

If you want to use the Neuron® for file storage or management from your desktop. you will need an FTP Client to access the Neuron®. FTP support in Windows is limited to unencrypted access, which is not permitted by the Neuron®. WinSCP is an FTP client that works well with the Neuron® and is commonly used on different platforms. When using WinSCP to connect, remember to use either explicit encryption on the FTP port (21 by default) or implicit encryption on the FTPS port (990 by default). SFTP is something else in WinSCP. You then connect, and once you are connected you can access the assigned folder, using the logical folder /.

Troubleshooting using sniffers

You can troubleshoot the FTP communication on the Neuron® in two ways: In the Program Data Folder, there exists an FTP subfolder, where communication logs will be stored in XML format, for 7 days. You can also create a web sniffer from the administrative page (under Communication/FTP Sniffer) that can be used to sniff on the communication for a maximum of an hour. The content of the files will not be displayed in these sniffers, only what connections are made, what commands are sent, and their responses.

#new, #neuron, #features, #api, #protocol, #ftp, #security, #mtls

Login-Auditor Exceptions

From build 2025-02-27 it is now possible to configure exceptions for endpoints in the Login Auditor that monitors all login attempts. The purpose of the Login Auditor is to detect malicious users and temporarily block them, and finally permanently block them, before they can gain access to the system by guessing user credentials.

The Login Auditor is configured in the Gateway.config file (also accessible under the Sources & Nodes and Gateway Configuration in the administration portal). The configuration consists of a sequence of growing intervals, and the number of allowed failed login attempts permitted from each remote endpoint. By creating an exception, you can provide another sequence of intervals and login attempt limits for those endpoints. If they are more permissible, they can be seen as white-listed to some extent.

Endpoint exceptions can be defined in three different ways:

By specifying the IP addresses explicitly, one at a time.
By specifying IP address ranges using CIDR format.
By specifying domain names, if the remote endpoints connect to the Neuron® using mTLS.

#new, #features, #neuron, #security

Developing for HTTPS with self-signed certificates

When developing on the Neuron® in a local environment, you typically don’t have access to a domain name accessible from the Internet. You can therefore not create a valid server certificate for the machine, something that is required for enabling encryption using protocols such as HTTP, XMPP, etc. If you do development that require such encryption, you can either choose to develop on a machine published on the Internet (not recommended), or temporarily use a self-signed certificate on your local version of the Neuron®, to enable encryption.

An example of task that requires use of HTTPS, is web access using HTTP/2. Using a browser to navigate your local Neuron®, typically results in the browser defaulting to HTTP/1.1. The original mechanism of upgrading a connection to HTTP/2 was obsoleted for unencrypted communication, so there’s no automatic manner for the browser to know it can upgrade. The upgrade mechanism typically used, requires use of TLS and ALPN, where the server states what protocols it supports. The browser therefore knows immediately which protocol to use, without having to perform a costly upgrade.

Another example that requires use of HTTPS, are connections that require authentication using mTLS.

For security reasons, uploading server certificates is not permitted in the GUIs of the Neuron®. Certificates are automatically created and maintained by the Neuron® using the ACME protocol, and certificate authorities on the Internet supporting the ACME protocol. The certificates are then maintained by the Neuron® itself and stored in its internal storage, to avoid the certificate being used elsewhere. To use a self-signed certificate therefore, we need to use script.

Checking if HTTPS is enabled

To check if HTTPS is enabled already on your machine, type the following in the script prompt:

Gateway.HttpServer.OpenHttpsPorts

If an empty array is returned, HTTPS is not enabled. If it contains one or more integers, they represent the port numbers you can use to access the Neuron® using HTTPS (i.e. HTTPS is enabled).

Creating a self-signed certificate

You can use the .NET runtime to create a self-signed certificate for your localmachine (localhost) for use in development. Type the following in a command-prompt, where you replace the FILENAME and PASSWORD with the filename you want to get the certificate exported to, and the password you want to use to protect the certificate:

dotnet dev-certs https -ep FILENAME -p PASSWORD --format Pfx

Configuring the Neuron®

As the admin interface does not permit uploading of custom certificates, we need to configure the Neuron® using script. The following example illustrates the properties that must be set:

DConfig:=DomainConfiguration.Instance;
DConfig.Domain:="localhost";
DConfig.UseDomainName:=true;
DConfig.UseEncryption:=true;
DConfig.PFX:=LoadFile(FILENAME);
DConfig.Password:=PASSWORD;
UpdateObject(DConfig);

For this script to have an effect, you need to use a broker with a build-time of 2024-12-12 or later.

Port configuration in Gateway.config

For the web server to open one or more HTTPS ports, they must be configured in gateway.config. From the Admin page, under Data and Sources & Nodes, you find the gateway configuration and port settings. There must be at least a port configuration for HTTPS, as illustrated in the following figure:

Restarting Neuron®

The configuration is applied when Neuron® is started. So for the changed to take effect, you need to restart the Neuron®. Once it has restarted, you can test the web server has opened at HTTP ports for incoming communication by executing the following script again:

Gateway.HttpServer.OpenHttpsPorts

Once you have confirmed the ports are open, you can direct a browser to the Neuron® using the domain localhost and the corresponding port, if different from the default HTTPS port 443.

Security Notice

Self-signed certificates should not be trusted, or installed or configured in the operating system to be trusted, as this would create a vulnerability that can be exploited. Instead, when navigating to the local development Neuron® using HTTPS, you will get a warning and the browser will initially refuse to show the contents. Accept the warning and tell the browser to view the content anyway.

Removing Certificate

To remove the certificate, and disable encryption on your local development server, you execute the following script:

DConfig:=DomainConfiguration.Instance;
DConfig.Domain:=null;
DConfig.UseDomainName:=false;
DConfig.UseEncryption:=false;
DConfig.PFX:=null;
DConfig.Password:=null;
UpdateObject(DConfig);

After executing the script, you need to restart the Neuron®.

#tutorial, #development, #neuron, #security

Login Auditor configuration in gateway.config

From build 2024-07-24, you can now configure the Login Auditor via the gateway.config file. This configuration is also accessible via the Node Explorer in the Admin menu (Sources & Nodes).

To configure the intervals of the Login Auditor, follow these steps:

Open the Admin page for the Neuron®.
Open the Node Explorer by clicking the Sources & Nodes button.
Expand the Gateway configuration folder.
Select the LoginAuditor node.
Edit, add or delete interval nodes accordingly (see figure below).
Restart the Neuron® when you’re done.

Note that:

Intervals are specified using the XML Duration format.
Interval nodes have no explicit Node IDs. For that reason, they are implicitly given GUIDs as IDs.

#new, #features, #neuron, #security

Controlling Multual TLS on a port level

As of build 2024-07-24 you can now control Mutual TLS settings (mTLS) on a Neuron® on a per port basis. You can for example use port 443 for normal HTTPS, without requesting client certificates, and enable mTLS on a secondary HTTPS port (for example 8088). To enable mTLS on only certain ports, do as follows:

Go to the admin page:
Open the Node Explorer by clicking the Sources & Nodes button.
Expand the Gateway Configiuration source.
Make sure the port is defined under Ports for use with the HTTPS protocol. (See figure below.)
Add a port-specific configuration node to the web server, specifying the same port to use mTLS. (See figure below.)
Restart Neuron®.

#neuron, #security, #new, #features

Posts tagged #security

No more posts with the given tag could be found. You can go back to the main view by selecting Home in the menu above.