Counter Reports

The Neuron® contains runtime counters of different types. They reside in the collection RuntimeCounters that contain a collection of RuntimeCounter objects. They are efficiently maintained in-memory, so incrementing them frequently does not affect performance greatly. Each counter is either persisted after one minute of no activity, or after 15 minutes of regular activity (at least one action per minute). Since the most recent counter value is often stored in-memory, the database collection is not the most reliable source of what counter values exist. The counter keys can be taken from the collection, but the most recent values should be retrieved using script. The script functions DecCounter, FlushClunters, GetCounter, GetCounters and IncCounter can be used to access runtime counters from script.

Note: RuntimeCounters is a namespace in the underlying code. This means that the SELECT statement select count(*) from RuntimeCounters will fail, since the RuntimeCounters reference will refer to the namespace instead of the collection. To make SELECT statements referring to counters, use double quotes around the name, to ensure the name is treated as a label, and not a reference to something else, as follows: select count(*) from "RuntimeCounters". You can also use the script functions.

Counter Script primer

Before delving into counter reports, first some background in how to extract information about counters using script. Each counter is represented by a RuntimeCounter object. You can select data from the database, either by referring to the class name of the counter object RuntimeCounter, or to the collection name, but using double-quotes to avoid referring to the namespace: "RuntimeCounters". (Should there be another class named RuntimeCounter in the code-behind, you need to refer to the fully-qualified name of the class, rather than the local name. Referring only to the local name would return a vector of the different class types matching the local name reference.)

Getting the properties avialable in persisted objects of a class, use the properties() function:

properties(RuntimeCounter)

This gives:

["ObjectId", "Key", "Counter"]

To extract available counters, we therefore do as follows: (Note here the use of the script function GetCounter() to get the value in memory, if any.

SELECT
	Key,
	GetCounter(Key) Value
FROM
	RuntimeCounter
ORDER BY
	Key

There will most probably be a lot of counters there. Some are communication-related, others are service-related. Others relate to specific Neuron-operations that may be of interest for statistical purposes or billing.

Counter differences

The script engine contains a feature referred to as subtraction of two dictionaries that makes it simple to compare counters from two different points in time. Since each key is unique, you can create a dictionary (object) of the counter values as follows:

Sample1:={};
foreach Counter in (SELECT Key FROM RuntimeCounter) do
	Sample1[Counter]:=GetCounter(Counter);

At a later time you similarly do:

Sample2:={};
foreach Counter in (SELECT Key FROM RuntimeCounter) do
	Sample2[Counter]:=GetCounter(Counter);

To get the differences of samples, you simply do:

Sample2-Sample1

Here, each object does not have to have the same propery values. If a property does not exist in one of the samples, but exists in the other, it is treated as having the zero element (in the underlying group, here being Integers).

Note: Once a set of counters is converted into a dictionary, as shown above, it can be persisted as a property itself in an object by itself in the database. The reports repsented below has this feature to save snapshots from the counters collection using labels that can be retrieved later for comparison purposes.

If many of the counters are the same between snapshots (i.e. there are many zeroes), we can remove all zeroes and create a reduced difference set as follows. First, we convert the dictionary into a vector, and then use the subset construction to create a smaller vector consisting of only objects whose values are not zero. By transposing this result, we get the result in tabular form.

Diff:=[foreach P in Sample2-Sample1 : P];
[P in Diff:P.Value!=0]T

In our example above, if Sample1 and Sample2 are made in relative close proximity, only a few counters have been modified. The result could look something like:

[[[XMPP.Server.Set.pubsub, 1]],
 [[XMPP.Server.Get.ping, 221]],
 [[E2EE.Counter, 11]]]

The example above shows 1 XMPP Publish/Subscribe operation, 221 XMPP ping operations to maintain connectivity and 11 End-to-End Encrypted stanzas communicated between samples.

Counter Labels

Samples in counter-reports described below are stored using Labels. These labels are stored in two different collections, depending on report: CounterLabels and BillableLabels. To get available labels, you can execute simple SELECT statements:

select Label from CounterLabels

or:

select Label from BillableLabels

Counter Reports

File-based reports make working with script easier. They allow you to parametrize script, and present the results in a more user-friendly manner (meaning, you don’t have to know script to know the results). There are two reports available that can be used to extract information from runtime counters. From Sources & Nodes in the administrative portal, you find the reports here:

Compare Counters Report

The Compare Counters report lets you compare two sets of samples with each other, taken at separate times. It also lets you label each sample. You select the counters you wish to compare by providing a Counter Key Prefix, which may or may not use a wildcard, which you can also specify. In the following example XMPP-related counters will be compared:

For our example, a brief reduced result set might look as follows:

Billable Counters Report

Certain counters could be referred to as billable counters, meaning, that some operators might base billing on server usage based on these counters. A special counter-comparison report exists that selects these counters automatically. All the operator needs to do is provide label names, to get suitable comparisons.

Example of what counters are included in the billable counters report, see the following report result comparing two snapshots close to each other (leaving results as zero):

The counters that appear have the following meaning:

Counter	Description
Broker.Accounts.Created	Number of XMPP accounts created.
KyC.*.IdentityApplication	Number of identity applications analyzed by KyC service *
Legal.Contract.Approved	Number of smart contracts approved.
Legal.Contract.BeingSigned	Number of contracts receiving digital signatures.
Legal.Contract.Failed	Number of contracts that failed.
Legal.Contract.Signed	Number of contracts reaching the signed state.
Legal.ID.Approved	Number of Identity Applications that were approved.
Legal.ID.Created	Number of Identity Applications that were created.
Legal.ID.Obsoleted	Number of Identity Applications that were obsoleted.
Legal.ID.Rejected	Number of Identity Applications that were rejected.
Legal.IDPreview.Approved	Number of Identity Preview Applications that where approved.
Legal.IDPreview.Created	Number of Identity Preview Applications that where created.
Legal.IDPreview.Obsoleted	Number of Identity Preview Applications that where obsoleted.
Legal.IDPreview.Rejected	Number of Identity Preview Applications that where rejected.
Legal.Template.Approved	Number of smart contract templates that were approved.
Legal.Template.Proposed	Number of smart contract templates that were proposed.
RemoteLogin.*	Number of Remote Login API authentication initiations made using authentication service *.

#counters, #neuron, #statistics, #reports, #billing

Broker Notification Configuration

As of build 2026-07-25, you can control what broker-events generate operator notifications, and which are only logged to the event log. It is a new system configuration page. This means the configuration will be part of the installation procedure of the Neuron as well. Each configuration have an associated environment parameter.

You find the new configuration page from the administration page, under Notarius Electronicus and Notifications.

Pressing the button (or opening the page via the drop-down menu), opens a page where individual event notifications can be enabled or disabled. For nodes with a lot of events, it might be more efficient and less distracting to disable certain notifications, and process them in bulk regularly instead of managing them individually.

Environment Variables

The following table lists environment variables that can be used to control the configuration of notifications:

Environment Variable	Description
BROKER_NOT_ACCOUNT_CREATED	If a notification should be sent when a new account is created.
BROKER_NOT_ACCOUNT_DELETED	If a notification should be sent when a new account is deleted.
BROKER_NOT_LEGAL_ID_RECEIVED	If a notification should be sent when a new a Legal ID application is received.
BROKER_NOT_OTHER_LEGAL_IDS	If a notification should be sent for every existing valid Legal ID that exists when a new Legal ID application is received.
BROKER_NOT_PEER_REVIEW_APPROVED	If a notification should be sent when a peer review of a Legal ID has been approved.
BROKER_NOT_LEGAL_ID_AUTO_APPROVED	If a notification should be sent when a Legal ID application has been automatically approved.
BROKER_NOT_LEGAL_ID_AUTO_REJECTED	If a notification should be sent when a Legal ID application has been automatically rejected.
BROKER_NOT_CONTRACT_PROPOSAL_RECEIVED	If a notification should be sent when a contract proposal has been received.
BROKER_NOT_ACCOUNT_RECOVERY_REQUEST	If a notification should be sent when an account recovery request has been received.
BROKER_NOT_API_KEY_CREATED	If a notification should be sent when an API key has been created.
BROKER_NOT_FEEDBACK_RECEIVED	If a notification should be sent when feedback has been received.
BROKER_NOT_ACCOUNT_REQUESTED	If a notification should be sent when an account has been requested.
BROKER_NOT_API_KEY_REQUESTED	If a notification should be sent when an API key has been requested.

#neuron, #admin

OAUTH 2 support in the Neuron

The Neuron® now supports OAUTH 2.0, and can be used as an authorization server in OAUTH-compliant systems (from build 2026-07-13). The OAUTH 2 environment available in the Neuron includes the following resources:

• An authorization resource (RFC 6749) at /oauth/authorize.
• A token resource (RFC 6749) at /oauth/token.
• A dynamic client registration resource (RFC 7591 and RFC 7592), supporting both public and confidential client registrations, at /oauth/register.
• A management resource for dynamic client registrations (RFC 7591) at /oauth/registration.
• A resource providing support for the device authorization flow (RFC 8628) at /oauth/device
• A token introspection resource, in accordance with RFC 7662 at /oauth/introspect
• A server OAUTH meta-data resource (RFC 8414) at /.well-known/oauth-authorization-server allowing external parties a way to find available resources and features.
• A resource providing meta-data for protected resources (RFC 9728) at /.well-known/oauth-protected-resource.

Apart from the resources defined, there are some notable features and extensions that are supported, and merit mentioning:

• Proof Key for Code Exchange by OAuth Public Clients (PKCE) (RFC 7636), securing the authorization flow.
• Support for refresh tokens (RFC 6749)
• Implicit token generation from traditional Neuron authentication, or Mutual TLS (mTLS).
• OAUTH clients (services) that want to use the Neuron as an OAUTH authorization server, should register themselves with the Neuron using the dynamic client registration interface, and provide a human-readable name, logotype and corresponding URIs for more information.
• Dynamic login forms generated by the OAUTH environment are generated first in Markdown, and then transformed to HTML before being returned to the user. This allows opertors of the Neuron to customize the look & feel of the login form, by customizing the MasterOAuth.md file available in the web root folder. The dynamic form is embedded in this master file, before being rendered as HTML.

Authentication, scopes and privileges are related as follows:

• Scopes in OAUTH, are translated into Privileges in the Neuron, having the prefix OAUTH.Scope. followed by the scope, where colons (:) are replaced by periods (.).
• An external party can authenticate itself with the OAUTH environment, and be authorized to receive a JWT token, which it should provide in subsequent requests using a Bearer token in an Authorization HTTP header.
• Dynamic client registrations, and clients that authenticate themselves using credentials for an XMPP account, have no privileges by default. Attempts to authorize access to specific scopes using such accounts will be rejected.
• Administrative user accounts will have the privileges provided to them via the Roles defined for the corresponding user accounts.

Dynamic client registration has been integrated into the Neuron using the following principles:

• The Neuron enables dynamic client registration by creating an API Key with the name OAUTH. If there is no such API key on the Neuron, dynamic client registration is not permitted.
• A remote endpoint can register at most 2 clients (public or confidential).
• Registered clients receive a corresponding XMPP account with the same client_id. These accounts are disabled for XMPP communication by default, but can be enabled manually. (Future work may provide a mechanism to automatically enable such accounts, by validating e-mail and/or phone numbers provided in client registration.)

Security Note: You can limit access to OAUTH using the Web-Application Firewall (or WAF), by restricting access to any resource that starts with /oauth/.

#new, #features, #neuron, #api, #oauth, #security

Queue Event Sink

A new Event Sink is available (from build 2026-06-01), permitting the temporary storage of logged events into a local Queue. Queues perist elements in an encrypted manner, and processes them in accordance with FIFO (First-In-First-Out) order. A queue frees processes that generate items from processes that process these items. They can work in different paces, and can be updated/run on different times.

To configure a Queue for temporary encrypted storage, you add a EventQueue element to the event sinks section of the Gateway.config file, and restart the Neuron®. You can also add such an event sink, via the administrative interface, under Sources & Nodes, and the Gateway configuration source.

You need to provide a Name for the Queue. This Name is later used to dequeue items that have been enqueued by the event sink. You also provide a number of days which items are stored in the queue. Older items are regularly purged from the queue, to avoid it growing indefinitely. A Sink ID must also be provided. This ID appears in event logs, in case something related to the event sink needs reporting.

To access enqueued events, you can either dequeue them in local services, or use the Queue API. If you are using Postman, there’s a public repository containing a Postman collection for the Queue API.

#new, #features, #neuron, #gateway, #events, #eventsink, #api, #queue

Using the Neuron to send and receive e-mail

The TAG Neuron® supports multiple protocols, among them, both the XMPP Protocol and the SMTP protocol. In XMPP, clients have accounts on brokers, and connect to their broker to connect to the federated XMPP network. Addresses in XMPP are called Jabber IDs, or JIDs for short. The Bare JIDs look like e-mail addresses, in the form: ACCOUNT@DOMAIN. Once connected, the connection is assigned a Full JID, in the form ACCOUNT@DOMAIN/RESOURCE.

The account can also interact using the SMTP protocol. This means, the client can send and receive e-mail seamlessly using the same XMPP account. E-mail messages sent to the account gets forwarded as messages to the client. The client can send messages back. The Neuron converts the messages into e-mail messages and forwards them to the recipient using the SMTP protocol.

There are some differences between how SMTP works on the Neuron®, and if you are using a traditional SMTP Server:

• In order to be able to communicate with a client on the Neuron, you need a presence subscription. This is a requirement on XMPP. SMTP does not have such a feature, which is one of the reasons why e-mail is so ripe with spam. The Neuron enforces presence subscription on e-mail as well. It does this via a squence of e-mails that are sent to anyone trying to send e-mail to an account on the Neuron. The e-mails will contain links allowing the sender to request presence. Once the client has accepted the presence subscription, the sender of the e-mail gets notified that it is able to send e-mail to the indented recipient.

• Outgoing messages are accumulated for a short time before being sent. In XMPP, messages are sent instantaneously to the recipient. If you use a chat program and write multiple rows of text, you want all rows to get into the same e-mail message. For this reason, the Neuron accumulates outgoing text messages, and forwards them once there is a silence for a minute (or an hour has passed since the first row).

• Incoming e-mail messages are sent to the client encoded in multiple ways. First, the text of the message is extracted, and forwarded as Markdown, HTML and/or plain text, as appropriate. The original contents of the e-mail is also forwarded, but can only be processed by the client, if it understands a separate e-mail extensions. This means, you can e-mail to any XMPP client, using text content. But only XMPP clients understanding the e-mail extension will be able to receive more advanced concepts, such as embedded attachments, etc.

• Outgoing messages from XMPP clients are typically of a simpler nature: Plain text, HTML and/or Markdown. This is transformed into multi-formatted e-mail messages accordingly, before being sent.

Example

As an example, following is a short interaction between an e-mail client using a hotmail e-mailing address, and an XMPP client, using an XMPP address. No Mail Relay was used on the Neuron used in this example.

Sending first mail to XMPP client

First step, that will trigger the approval process (or white-listing process), is to send an e-mail to the XMPP client you want to interact with. The contents of this e-mail will be ignored; it can be anything. When the Neuron receives this e-mail, it will check the roster of the recipient to see if an approved presence subscription is available. If not, it will immediately return an e-mail stating this fact.

Sending a presence subscription request

The e-mail will contain a link the recipient can click on, that will send a presence subscription request to the intended recipient. The link is cryptographically protected with the associated e-mail, and the sender and receiver identities. The sender will also receive an e-mail confirmation that the presence subscription request was sent to the recipient.

Receiving presence subscription acceptance confirmation

If the recipient accepts the presence subscription request, a two new e-mails are sent to the original sender, confirming this fact, and instructing the sender that it is now allowed to send e-mails to the intended recipient. The first e-mail shows the sender has been added to the white-list (roster in XMPP). The second, that the presence subscription request has been accepted.

Testing e-mail <-> XMPP link

The e-mail sender should now be able to communicate with the XMPP recipient. Sending a simple test message to the recipient, will make the text in the e-mail appear as a chat message in the XMPP client. The XMPP client can respond using chat, just as if the sender was another XMPP-client.

The XMPP chat response is transformed into an e-mail and sent to the original sender.

Troubleshooting SMTP Communication

A lot of problems can occur in SMTP communication. The principle reason is lack of cybersecurity in the original specifications, and security added on-top, as people became aware of threats. There is often no clear indication as to why there are problems sending or receiving e-mail.

The principle resource for troubleshooting e-mail is SMTP communication logs. These are stored in the SMTP subfolder in the Neuron Program Data Folder. The SMTP folder contains subfolders with communication logs for different senders and recipients. Input logs (containing information about incoming messages) are stored separately from Output logs (containing information about outgoing messages). To distinguish the two, Input logs are suffixed with IN and Output logs are suffixed with OUT.

When troubleshooting incoming logs, you need to know from where the message will be received. This can be learned from DNS SPF entries. When troubleshooting outgoing logs, you need to know to where the message will be sent. This can be learned from DNS MX entries. There are tools online for finding out both. You can also use the Script engine on the Neuron to get the relevant DNS records.

Also, you need to ensure the DNS has been configured correctly for the Neuron, especially those related to Sender Policy Framework SPF (TXT records) and Mail Exchange (MX records).

#tutorial, #neuron, #smtp, #email, #mail, #xmpp