Troubleshooting Stack Overflow and Out of Memory Errors

The Neuron® normally runs as a Windows Service using the limited Local Service user account. Any errors in the Neuron® or any of the applications hosted within it, are normally logged to the internal event log, which distributes events to registered event sinks. These may record them in files, internal databases, the Windows Event Log, or forward them to external sources, both for real-time monitoring or storage. Communications can also be monitoried and trouble-shooted using sniffers that can be temporarily attached to different communication channels. Any Exception occurring within the system can also be logged separately together with stack traces, and statistics of such is generated when the Neuron® restarts. But one type of Exception is particularly difficult to troubleshoot, especially for a Windows Service that runs invisibly on a server: Stack Overflow, Out-of-Memory, Access Violation Exceptions and other fatal exceptions can typically shut down the service immediately, without any possibility to record details about the event for troubleshooting. The only notification an operator may receive, is that the Neuron® restarts unexpectedly. After reviewing the backups folder, the operator may also notice the database has been repaired, since the Neuron® was shut down ungracefully (i.e. the process was killed). There are a couple of ways you can find out what causes such an error.

Remote Debugging

One way to find this error, is to attach to the Neuron® process on the server using Visual Studio Remote Debugging tools. Once attached to the process, the debugger can be configured to break when the Exception occurs, giving you a chance to examine the stack trace before the process is killed. There are difficulties with this approach however. One is that the code is probably optimized for Release, and difficult to Debug. You must also open ports, install the remote debugger, etc.

Running as a Console

Another way to troubleshoot the Exception, is to run the console version of the Neuron® in a Terminal Window or Command Prompt. The difficulty here, is to make sure the Terminal Window is using the correct user privileges to mimic the environment the Windows Service has. This can be done by downloading PSTools from Microsoft. It provides tools to access internal features in the Windows operating system. Once installed, you open a Terminal Window (Command Prompt) with administrative privileges. You go to the folder where PSTools is installed, and you type the following command:

psexec -i -s -u "NT AUTHORITY\LocalService" cmd.exe

This will open a new terminal window with the correct user privileges. Before starting the Neuron® in the new terminal window, you need to stop and disable the Windows Service, so it does not interfere. Then, in the new terminal window, go to the installation folder of the Neuron®, and type:

Waher.IoTGateway.Svc.exe -console

It uses the same executable file that is executed by the Windows Service, but the -console switch starts it in console mode. This will enable terminal output of the internal event log, and any system message that will appear. If a Stack Overflow, Out of Memory or Access Violation Exception occurs (or any other fatal type of Exception), this information will be output to the terminal window as the process is killed.

#tutorial, #troubleshooting

New Neuron Popup System

We have replaced window.alert, window.confirm, and window.prompt with a custom popup system. This provides better control and accessibility for user interactions.

Overview

• Improved User Experience: The popups have a consistent style and can be customized.
• Accessibility: Focus trapping ensures proper keyboard navigation.
• Better Control: Supports handling of modal dialogs with promises.

How to use it

The popup system is located in the Master.js file. To use it include the folowing headers in your markdown or master file.

Javascript: /AlertPopup.md.js
Javascript: /ConfirmPopup.md.js
Javascript: /PromptPopup.md.js
Javascript: /Master.js

You also need a dialog container in the bottom of your html body.

<div id="native-popup-container"></div>

Additionally, the theme used must implement the base theme base.cssx for styling. Alternatively you could style it yourself, but it would be recomended to instead cascade over the already existing css or changing the themes cssx variables;

Usage

Alert

Use Alert(message) to display a basic alert popup.

const popup = PopupHandler();
await popup.Alert("This is an alert message.");

Confirm

Use Confirm(message) to display a confirmation dialog that returns a boolean.

const result = await popup.Confirm("Are you sure?");
if (result) {
    console.log("User confirmed.");
} else {
    console.log("User canceled.");
}

Prompt

Use Prompt(message) to capture user input.

const userInput = await popup.Prompt("Enter your name:");
console.log("User entered:", userInput);

Details

• All popups are contained in a shared container.
• The popup backdrop is generated by the Master.js file, and placed at the bottom of the body tag
• If you initiate a new popup whilst one is already open, it will cover the old one. The old one will be accesible after you interact with the newly created one. You can think of the popups in a stack datastructure.
• Focus trapping disables focusing outside the popup using tab and shift + tab.
• Your master markdown file must include the following element at the bottom to contain the popups:
• If you dont need to wait for an Ok press when using Alert() you don’t need to await the response.

#neuron, #api, #javascript, #tutorial

Developing for HTTPS with self-signed certificates

When developing on the Neuron® in a local environment, you typically don’t have access to a domain name accessible from the Internet. You can therefore not create a valid server certificate for the machine, something that is required for enabling encryption using protocols such as HTTP, XMPP, etc. If you do development that require such encryption, you can either choose to develop on a machine published on the Internet (not recommended), or temporarily use a self-signed certificate on your local version of the Neuron®, to enable encryption.

An example of task that requires use of HTTPS, is web access using HTTP/2. Using a browser to navigate your local Neuron®, typically results in the browser defaulting to HTTP/1.1. The original mechanism of upgrading a connection to HTTP/2 was obsoleted for unencrypted communication, so there’s no automatic manner for the browser to know it can upgrade. The upgrade mechanism typically used, requires use of TLS and ALPN, where the server states what protocols it supports. The browser therefore knows immediately which protocol to use, without having to perform a costly upgrade.

Another example that requires use of HTTPS, are connections that require authentication using mTLS.

For security reasons, uploading server certificates is not permitted in the GUIs of the Neuron®. Certificates are automatically created and maintained by the Neuron® using the ACME protocol, and certificate authorities on the Internet supporting the ACME protocol. The certificates are then maintained by the Neuron® itself and stored in its internal storage, to avoid the certificate being used elsewhere. To use a self-signed certificate therefore, we need to use script.

Checking if HTTPS is enabled

To check if HTTPS is enabled already on your machine, type the following in the script prompt:

Gateway.HttpServer.OpenHttpsPorts

If an empty array is returned, HTTPS is not enabled. If it contains one or more integers, they represent the port numbers you can use to access the Neuron® using HTTPS (i.e. HTTPS is enabled).

Creating a self-signed certificate

You can use the .NET runtime to create a self-signed certificate for your localmachine (localhost) for use in development. Type the following in a command-prompt, where you replace the FILENAME and PASSWORD with the filename you want to get the certificate exported to, and the password you want to use to protect the certificate:

dotnet dev-certs https -ep FILENAME -p PASSWORD --format Pfx

Configuring the Neuron®

As the admin interface does not permit uploading of custom certificates, we need to configure the Neuron® using script. The following example illustrates the properties that must be set:

DConfig:=DomainConfiguration.Instance;
DConfig.Domain:="localhost";
DConfig.UseDomainName:=true;
DConfig.UseEncryption:=true;
DConfig.PFX:=LoadFile(FILENAME);
DConfig.Password:=PASSWORD;
UpdateObject(DConfig);

For this script to have an effect, you need to use a broker with a build-time of 2024-12-12 or later.

Port configuration in Gateway.config

For the web server to open one or more HTTPS ports, they must be configured in gateway.config. From the Admin page, under Data and Sources & Nodes, you find the gateway configuration and port settings. There must be at least a port configuration for HTTPS, as illustrated in the following figure:

Restarting Neuron®

The configuration is applied when Neuron® is started. So for the changed to take effect, you need to restart the Neuron®. Once it has restarted, you can test the web server has opened at HTTP ports for incoming communication by executing the following script again:

Gateway.HttpServer.OpenHttpsPorts

Once you have confirmed the ports are open, you can direct a browser to the Neuron® using the domain localhost and the corresponding port, if different from the default HTTPS port 443.

Security Notice

Self-signed certificates should not be trusted, or installed or configured in the operating system to be trusted, as this would create a vulnerability that can be exploited. Instead, when navigating to the local development Neuron® using HTTPS, you will get a warning and the browser will initially refuse to show the contents. Accept the warning and tell the browser to view the content anyway.

Removing Certificate

To remove the certificate, and disable encryption on your local development server, you execute the following script:

DConfig:=DomainConfiguration.Instance;
DConfig.Domain:=null;
DConfig.UseDomainName:=false;
DConfig.UseEncryption:=false;
DConfig.PFX:=null;
DConfig.Password:=null;
UpdateObject(DConfig);

After executing the script, you need to restart the Neuron®.

#tutorial, #development, #neuron, #security

PlantUML integration on Mac OS

PlantUML is a tool that can generate various types of diagrams from textual definitions. The TAG Neuron® can be integrated with PlantUML, if it is installed on the same machine. This article outlines how to install PlantUML on your MAC so that it will work with the TAG Neuron® installed on the same MAC machine.

1. First, make sure you have installed Java on your machine. You can easily do this from a terminal window by typing:

   java -version
   

   If not available, make sure to install it first.

2. Download the plantuml.jar file, using the license that suits your requirements. This contains the compiled Java program that converts PlantUML text files into image diagram files.

3. The downloaded file will contain version information included in the file name. For the file to be found by the TAG Neuron®, you need to rename the downloaded file, to just plantuml.jar.

4. Check what folders are in your system PATH. You can do this from the terminal window by executing:

   echo $PATH
   

5. Copy the plantuml.jar file downloaded earlier into one of the folders in defined in PATH, for example /opt/local/bin. As long as it is located in a folder defined in PATH, the TAG Neuron® will find it when it restarts.

6. Restart the Neuron®

After restarting the Neuron®, you should se an event logged, that confirms that PlantUML and Java has been found on the machine, and that the PlantUML Markdown integration is active:

PlantUML found. Integration with Markdown added.
  Timestamp=01-06-2024 12:08:35PM, Level=Minor, Path=/opt/local/bin/plantuml.jar, Java=/usr/bin/java

You can later update PlantUML by simply downloading a new plantuml.jar file, replacing the previous file. You do not need to restart the Neuron® after updating PlantUML.

To test the PlantUML integration, you can open the PlantUML Lab from the Admin menu by pressing:

A page opens with two panes. In the left one you can edit PlantUML text. In the right one you’ll see the rendering of the text (or any errors found):

You can also check the PlantUML integration section in the Markdown documentation to see that the integration works.

#documentation, #tutorial, #neuron, #plantuml, #mac

Graphviz integration on Mac OS

The TAG Neuron® can integrate with GraphViz, permitting automatic generation of diagrams from GraphViz definitions in Markdown. The integration requires GraphViz to be installed by the operator of the Neuron®. Follow these steps to install GraphViz:

1. On Mac OS, GraphViz can be installed using MacPorts. It is an application that simplifies the installation of open source projects on your Mac via the terminal window. The MacPorts project provides multiple installers for different versions of Mac OS.

2. Once MacPort is installed, install GraphViz from the terminal window, using the following command-line (as described in the GraphViz MAC installation documentation).

   sudo port install graphviz
   

   If the terminal session cannot find port, use the full path of the MacPort application as follows:

   sudo /opt/local/bin/port install graphviz
   

3. Similarly, you can install a GUI tool for viewing GraphViz documents:

   sudo port install graphviz-gui
   

   Or:

   sudo /opt/local/bin/port install graphviz-gui
   

Once GraphViz has been installed on the Mac, you need to restart the Neuron®. Once restarted, you should see a log entry during startup confirming the Neuron® has found GraphViz, and that the integration is in effect:

GraphViz found. Integration with Markdown added.
  Timestamp=31-05-2024 8:17:17PM, Level=Minor, Installation Folder=/opt/local/bin, Binary Folder=/opt/local/bin, dot=True, neato=True, fdp=True, sfdp=True, twopi=True
  circo=True

From the Admin-menu, you can now open the GraphViz Lab:

Once open, you can edit your diagrams in the left pane, and view the corresponding output in the right pane:

Integration into existing Markdown pages will also show that the integration works.

#tutorial, #documentation, #neuron, #graphviz, #mac