Neural Networks of Societies of Autonomous Agents

David Luckham

In a previous article we discussed how a society of autonomous agents (AA’s) could be organized in a hierarchy to implement a large scale mission such as a company’s introduction of a new product to market, or a government’s handling of a natural disaster. In such cases a mission would have an outline of actions to be performed. The outline would be hierarchical with all actions, both low level and higher level, being performed by AAs.

The organization of this hierarchy of communicating AAs now becomes an issue. We give a detailed example of the definition of a hierarchical society of AAs and its organization by means of a neural network.

We assume that AAs are small single purpose programs that execute when needed and can communicate with other AAs. Also, that the communication abilities of each AA are part of its specification.

Example: A company initiates an event-driven sales campaign for a new product using a hierarchical mission flow implemented by AAs. It is decided to organize the mission into four layers, with each layer containing strategies to use information abstracted from lower layers to track customer behavior, classify customers according to qualification criteria, and deploy strategies to engage customers that are qualified.

The challenge is to create a matching hierarchy of AAs that implements this this event-driven sales campaign. The AAs in all four layers will have different roles. All AAs must be able to communicate with each other and with customers by means of events both within a layer and with layers immediately above and below.

Let’s break down the roles and communication at each level. The goal is to ensure that each layer’s agents abstract the lower-level interactions, creating higher-level events that track customers behavior, classify customers and progressively engage customers that become qualified.

The Four Level Mission Implementation by AAs

Layer 1: Customer Engagement Triggers

This is the foundational layer, where AAs detect and respond to low-level interactions, such as:

Page visits
Clicks on ads, product pages, or links
Social media interactions
Email opens

Role of AAs:

Detection Agents: Each agent monitors specific types of user interactions in real-time. For example, one agent could track clicks on product pages, while another focuses on social media engagement.
Response Agents: These agents react based on pre-defined rules, such as sending a follow-up email or displaying a personalized offer based on the interaction.

Communication:

Detection agents will pass relevant data (e.g., time, context of the action) to the agents in Layer 2 for aggregation and qualification.

Layer 2: Customer Qualification

In this layer, agents analyze and aggregate the customer’s low-level interactions (triggered in Layer 1) to prioritize leads. The focus is on evaluating customer intent and potential value.

Role of AAs:

Aggregation Agents: These agents collect data from multiple touchpoints (e.g., product page views, time spent on site, multiple clicks) and build a comprehensive view of the customer’s interest.
Evaluation Agents: They assess the aggregated data to prioritize customers based on predefined criteria, such as frequency of interaction, time spent on key pages, or interactions with high-value content.

Communication:

Once a customer is qualified (e.g., a high likelihood of purchasing), the evaluation agents trigger communication with the agents in Layer 3. Data about the customer’s preferences and interaction history is passed along.

Layer 3: Personalized Engagement

This is where AAs take action based on the qualified leads from Layer 2, delivering personalized content and offers to engage the customer more deeply.

Role of AAs:

Personalization Agents: These agents generate personalized marketing messages or offers based on the customer’s previous actions. For example, they could offer a discount on a product the customer has repeatedly viewed or customize a promotional email based on product preferences.
Engagement Agents: They send the personalized messages, whether through email, SMS, in-app notifications, or social media retargeting.

Communication:

The agents in this layer receive continuous feedback from Layer 2 about the customer’s reactions to the personalized content (e.g., clicks on offers, further interactions), which are then either reported back to Layer 2 for refinement or abstracted and moved forward to Layer 4.

Layer 4: Customer Retention and Loyalty

At this final layer, AAs focus on maintaining customer satisfaction and fostering long-term loyalty based on their behavior and engagement history.

Role of AAs:

Retention Agents: These agents monitor long-term engagement patterns, customer satisfaction, and churn risks. They analyze customer feedback, purchase frequency, and post-purchase engagement.
Loyalty Agents: They implement reward programs, special offers, and incentives based on customer loyalty data. This could include loyalty points, exclusive discounts, or early access to new products.

Communication:

The agents here continue to monitor the customer’s interaction and pass updates back to the agents in Layers 2 and 3, ensuring that ongoing personalized engagement is based on the evolving relationship.

Efficiency Through Abstraction

Each layer in this hierarchy abstracts the complexity of the lower levels, reducing the need for manual oversight and enabling the AAs to dynamically respond to changing customer behaviors. For instance:

Layer 1 abstracts individual actions into customer behavior data.
Layer 2 abstracts sets of behaviors into qualified leads.
Layer 3 abstracts the results of personalized engagement strategies based on qualified leads.
Layer 4 abstracts ongoing customer interactions into long-term retention strategies.

The communication flow between agents ensures that the system is both efficient and responsive, enabling real-time decision-making and actions tailored to each customer’s behavior as they move through the layers.

Figure 1: ChatGTP-4’s simple depiction of the 4-layer hierarchy. The numbering of the layers is wrong.

Organization As an Artificial Neural Network of AAs.

The four-level AA hierarchy will be organized to resemble an artificial neural network (ANN). This would involve treating the autonomous agents (AAs) as neurons in the network, while the communication between layers would be managed via synapse-like connections.

Structure of the ANN-Based Hierarchy

In this model, the four layers of the AA hierarchy can be structured as different layers in an ANN, where each layer corresponds to specific customer engagement abstraction:

Layer 1: Customer Engagement Triggers

Acts as the input layer of the ANN.
AAs in this layer function as input neurons, detecting low-level customer interactions (clicks, page visits, etc.).
These “neurons” fire signals when specific triggers occur, passing data to the next layer via synapses that connect the neurons in that layer.
Each “neuron” or AA in this layer monitors for a different type of customer activity (or event), translating raw interactions into signals that are fed forward.

Layer 2: Customer Qualification

Functions as the first hidden layer in the ANN.
The AAs here serve as hidden neurons that aggregate and transform sets of customer behaviors into qualified leads.
This layer receives input from the Layer 1 neurons, processes it, and generates an output (e.g., lead scores).
The neurons use activation functions that determine the strength of qualification based on weights assigned to different behaviors.

Layer 3: Personalized Engagement

Serves as the second hidden layer in the ANN.
AAs act as neurons that personalize engagement strategies based on the signals (qualified leads) from Layer 2.
The AAs apply weights to these signals, adjusting the response based on customer profiles and preferences.
The output here is a set of personalized marketing strategies or offers.

Layer 4: Customer Retention and Loyalty

Acts as the output layer of the ANN.
AAs function as output neurons that generate retention strategies based on signals from Layer 3.
The output neurons activate based on long-term interaction patterns, triggering loyalty programs, rewards, or churn prevention strategies.

Communication via Synapse-like Connections

In this ANN-inspired organization, the communication between AAs simulates synapse-like connections:

Weighted Connections: Just as in ANNs, connections between AAs can be weighted to represent the importance of each signal. For example:
- A click on a product page may have a higher weight than a general page visit.
- A lead score may be weighted higher if the customer has interacted frequently over a short period.
Activation Functions: AAs can apply activation functions to determine whether to pass a signal forward.
- For instance, a certain level of interaction might be required to trigger engagement at Layer 3.
- Similarly, Layer 4 AAs could activate only when retention thresholds are reached.

Figure 2: ChaptGTP-4 diagram of a 4-Layer Neural Network of AAs. This figure does not look like a neural network, particularly the synapses for communication, and is not what the author would have drawn.

DISCUSSION

How the Article was written. This article was written by ChatGTP-4 under supervision as much as possible by the author. Prompts often had to be refined several times, which reflects a learning curve in working with these systems to produce an acceptable end product.

A more detailed article that explains how the communication between neurons (agents) over synapse-like connections actually works, is forthcoming.

ChatGTP-4. was asked what it was, here is its answer:

“I am ChatGPT-4 with the DALL·E 3 model integrated for image generation. My current version is a part of OpenAI’s GPT-4 lineup, fine-tuned for various tasks like generating images, creating diagrams, browsing the internet (if needed), and assisting with detailed information.”

Assessment. ChatGTP-4 is pretty good at producing relevant text. I found its answers to be a good outline of a task’s implementation, but lacking in essential details such as how aggregation of observed customer behavior is done, methods used, etc.

The issue of training a neural network of AAs was not raised or tackled by ChatGTP-4. It did not seem to understand that this Neural network of communicating AAs might not be successful at customer retention, and that training would be an issue.

Also. in the area of creating diagrams,, it was a total dunce on day 1. It actually offered to draw a diagram of the 4-layer hierarchy. What transpired was that ChatGTP-4 did not understand even the simplest spatial concepts such as above, below, top, bottom, rotate (left or right) inside, outside, and so on. Asking it to simplify a diagram was a disaster leading to anything but a simplification. I spent many hours trying to get it to draw a simple outline diagram of the four-layer hierarchy. It never could get four layers from bottom to top in order 1-2-3-4.

One week later its ability to draw simple diagrams had somewhat improved. And the resulting improved diagrams are shown here. Also, its understanding of spatial concepts had really progressed. The current version is much better. But it still cannot edit a diagram to do something simple to it, like change the numbering, without totally changing the diagram.

My overall assessment is that ChatGTP-4 has improved in one week. Perhaps this is a version update. But My assessment is that ChatGTP-4 is simply a smart search engine that regurgitates stored text. And if it is to become the system that is causing Geoffrey Hinton concern, that won’t happen any time soon. He should have no worries that ChatGTP-4 is going to take over the world. Not in our lifetimes!