From Dashboards to Decisions: Building an AI-Powered Microservices Troubleshooting System

Using Strands Agents, CloudWatch and a locally hosted Llama 3.1 8B model on Amazon EC2

What if your monitoring system could do more than tell you that something is broken?
What if it could investigate the evidence, connect related failures, explain what is likely happening, recommend the next action — and, for carefully controlled cases, even remediate the problem?

This is the problem I wanted to explore in my microservices environment.

Collecting data is easy. Turning that data into a useful engineering decision is much harder.

The next question naturally followed:

Can we use an AI agent to reduce the amount of manual investigation engineers need to perform after an alert?

This article describes how I approached that problem using Strands Agents, AWS CloudWatch, Streamlit, and a locally hosted Llama 3.1 8B model running on Amazon EC2.

The model was deliberately not a paid API.

More importantly, this experiment wasn’t just about adding an LLM to CloudWatch. It was about exploring what happens when we introduce an agentic investigation and remediation layer into a real microservices architecture.

Situation: We already had observability

In a microservices environment, observability produces a tremendous amount of information.

Depending on the platform, we may have:

multiple microservices
ECS workloads
CloudWatch logs
metrics
alarms
dashboards
deployment information
application errors
dependency failures
queue activity
infrastructure events

On paper, this gives engineers everything they need to troubleshoot an incident.

But there is an important distinction:

Having the data doesn’t mean the problem is easy to diagnose.

A typical incident might look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Alert
  │
  ▼
Open dashboard
  │
  ▼
Identify affected service
  │
  ▼
Open CloudWatch
  │
  ▼
Search logs
  │
  ▼
Find errors
  │
  ▼
Read surrounding events
  │
  ▼
Check dependent services
  │
  ▼
Correlate timestamps
  │
  ▼
Look at recent deployments
  │
  ▼
Form a hypothesis
  │
  ▼
Decide what to do

The dashboard might tell me:

orders-service error rate has increased.

CloudWatch might tell me:

Connection timeout

But the engineer still has to ask:

Is this the root cause or a symptom?
Which dependency is failing?
Did this start after a deployment?
Are other services seeing the same problem?
Has this happened before?
What should I investigate next?
Can anything safely be remediated?

That investigation can consume a significant amount of engineering time.

The problem wasn’t a lack of observability.

The problem was the investigation loop.

Task: Move from detection to investigation

I wasn’t trying to build another monitoring dashboard.

The objective was to introduce an AI-assisted investigation layer on top of the existing observability platform.

The intended workflow was:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
                 ┌─────────────────┐
                 │    Developer    │
                 └────────┬────────┘
                          │
                          ▼
                  "Investigate issue"
                          │
                          ▼
                 ┌─────────────────┐
                 │   Strands Agent │
                 └────────┬────────┘
                          │
             ┌────────────┼────────────┐
             ▼            ▼            ▼
        CloudWatch      Metrics     Context
           Tools         Tools       Tools
             │            │            │
             └────────────┼────────────┘
                          ▼
                       Evidence
                          │
                          ▼
                    Local LLM
                          │
                          ▼
                Investigation Report

The objective wasn’t to replace engineers.

It was to automate the repetitive first stage of troubleshooting:

Collect the evidence → correlate it → summarize it → form an initial hypothesis.

The engineer would still make the final decision.

Action: Introduce an agent into the existing architecture

Rather than building a completely separate AI platform, I wanted the agent to sit naturally on top of the existing microservices and observability architecture.

The initial design was:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
                    Microservices
                         │
                         ▼
                    CloudWatch
                         │
                 ┌───────┴───────┐
                 │               │
                Logs           Metrics
                 │               │
                 └───────┬───────┘
                         ▼
                    Agent Tools
                         │
                         ▼
                   Strands Agent
                         │
                         ▼
                     Local LLM
                         │
                         ▼
                  Investigation
                         │
                         ▼
                     Developer

This is where Strands Agents became particularly interesting.

The AWS Builder CloudWatch Analyzer example demonstrates a similar architecture: a Strands Agent uses tools to retrieve CloudWatch information and produce an analysis. The reference implementation uses a managed model through Amazon Bedrock. AWS Builder — Automating CloudWatch Log Analysis with Amazon Strands Agent

I wanted to take that idea in a different direction.

The model didn’t need to be a paid API

The obvious architecture would have been:

1
2
3
4
5
6
7
Strands Agent
      │
      ▼
Amazon Bedrock
      │
      ▼
Foundation Model

That is a perfectly valid production architecture.

But I had another objective.

I wanted to learn more than simply:

“How do I call an LLM API?”

I wanted to understand:

“What does it take to actually operate a model as part of an application?”

So I deliberately replaced the managed model layer with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
Strands Agent
      │
      ▼
    Ollama
      │
      ▼
Llama 3.1 8B
      │
      ▼
   Amazon EC2

The resulting architecture became:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
                    Developer / UI
                          │
                          ▼
                  ┌───────────────┐
                  │ Strands Agent │
                  └───────┬───────┘
                          │
          ┌───────────────┼───────────────┐
          ▼               ▼               ▼
    CloudWatch Tool   Metrics Tool   Context Tool
          │               │               │
          └───────────────┼───────────────┘
                          ▼
                       Evidence
                          │
                          ▼
                    Ollama / EC2
                          │
                          ▼
                   Llama 3.1 8B
                          │
                          ▼
                Investigation Summary

This separation is important.

Strands handles the agent orchestration.

Tools provide access to operational data.

Python handles deterministic processing and calculations.

Ollama serves the model.

Llama 3.1 8B performs reasoning and synthesis.

CloudWatch remains the source of operational truth.

Why I didn’t simply send all the logs to the LLM

A naive implementation could look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
CloudWatch
    │
    ▼
Dump logs
    │
    ▼
LLM
    │
    ▼
"Tell me what's wrong"

I didn’t want that architecture.

Production logs are noisy.

They contain:

repeated messages
health checks
warnings
stack traces
retries
unrelated events
duplicate errors
enormous amounts of contextual information

Sending everything to the model creates three problems:

More data to process
More latency
More opportunity for the model to get distracted

Instead, I treated CloudWatch as a set of tools available to the agent.

The agent could retrieve the information it needed.

Conceptually:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
Agent
 │
 ├── Get recent errors
 │
 ├── Inspect surrounding log events
 │
 ├── Check affected service
 │
 ├── Check related dependency
 │
 └── Inspect additional context
 │
 ▼
Evidence

That distinction is what turns a simple LLM application into an agentic workflow.

Tools are the bridge between the agent and reality

The agent doesn’t need to understand how CloudWatch works internally.

It needs capabilities.

For example:

1
2
3
4
5
6
7
8
@tool
def get_cloudwatch_logs(
    log_group: str,
    start_time: str,
    end_time: str,
    filter_pattern: str | None = None
):
    ...

Conceptually, the agent sees:

1
2
3
4
5
6
Available capabilities:

get_cloudwatch_logs()
get_service_metrics()
get_deployment_context()
get_service_status()

This gives us an important architectural boundary:

1
2
3
4
5
6
7
8
9
                 Agent
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
      Logs      Metrics    Context
        │          │          │
        └──────────┼──────────┘
                   ▼
                Evidence

If I later want to add:

deployment history
GitHub commits
runbooks
service ownership
incident history
Kubernetes information

I can add capabilities without redesigning the entire application.

Deterministic processing before AI reasoning

Another important design decision was keeping deterministic work outside the LLM.

For example, suppose CloudWatch contains thousands of log events.

Python can perform:

filtering
aggregation
counting
timestamp normalization
grouping
error classification
threshold calculations

The model doesn’t need to calculate:

“There were 142 errors.”

Python can calculate it.

The model can then reason about what those 142 errors mean.

The pipeline becomes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
Raw Logs
   │
   ▼
Python
   │
   ├── Filter
   ├── Group
   ├── Count
   ├── Normalize
   └── Validate
   │
   ▼
Structured Evidence
   │
   ▼
Strands Agent
   │
   ▼
Llama 3.1 8B
   │
   ▼
Interpretation

This became one of the most important principles of the implementation:

Don’t ask the LLM to do something deterministic that Python can do reliably.

From errors to causal chains

Consider a simplified incident:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
orders-service
      │
      ▼
HTTP 500
      │
      ▼
payment-service timeout
      │
      ▼
database connection failures

A dashboard might display all three.

But the engineer has to correlate them.

The agent can help construct a hypothesis:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Observed
────────
orders-service HTTP 500
payment-service timeout
database connection failures

        │
        ▼

Likely relationship
───────────────────
Database connection
pool exhaustion

        │
        ▼

Potential impact
────────────────
payment-service becomes
unresponsive

        │
        ▼

Downstream symptom
──────────────────
orders-service returns 500

Notice the language:

Observed

versus

Likely

versus

Potential

This distinction matters.

I don’t want an LLM saying:

“The root cause is definitely X.”

when the available evidence only supports:

“X is currently the most likely explanation.”

That is an important safety boundary.

The output should be useful to an engineer

The goal isn’t to produce a beautiful AI paragraph.

A useful investigation report should answer:

What happened?

1
orders-service error rate increased by 38%.

What evidence supports it?

1
2
3
4
142 errors detected in the last 15 minutes.

96 were connection timeouts against
payment-service.

What is the likely cause?

1
2
payment-service appears to be experiencing
connection exhaustion.

What is uncertain?

1
Database saturation has not yet been confirmed.

What should I investigate next?

1
2
3
1. Check database connection utilization.
2. Inspect payment-service deployment history.
3. Compare connection failures across instances.

That is much more useful than:

“There seems to be a problem with payment-service.”

The first result: reducing the investigation loop

The practical difference is the workflow.

Before

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
Alert
 ↓
Find service
 ↓
Open CloudWatch
 ↓
Search logs
 ↓
Read stack traces
 ↓
Check dependencies
 ↓
Check deployment
 ↓
Correlate timestamps
 ↓
Form hypothesis

After

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Alert
 ↓
Run investigation
 ↓
Agent
 ↓
Tools
 ↓
Evidence
 ↓
Analysis
 ↓
Engineer validates

The engineer still makes the decision.

But much of the repetitive information gathering is automated.

Then came the obvious question

Once the agent could investigate a problem, I asked:

If the agent knows what is wrong, why not let it fix it?

Technically, there is nothing preventing us from giving an agent tools that can modify AWS resources.

For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Agent
 │
 ├── Inspect logs
 ├── Identify issue
 ├── Determine remediation
 │
 ▼
Remediation Tool
 │
 ├── Restart ECS task
 ├── Scale service
 ├── Disable unhealthy target
 └── Redrive messages

And this is where the project became much more interesting.

Because agentic remediation introduces a completely different risk profile.

Investigation is not the same as remediation

There is a huge difference between:

“The service appears unhealthy.”

and:

“I restarted 50 production tasks.”

The first is an analytical conclusion.

The second is a production action.

An agent that can read has a very different risk profile from an agent that can write.

I therefore started thinking about tools in two categories:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
                  Agent Tools
                       │
             ┌─────────┴─────────┐
             ▼                   ▼
         READ TOOLS          WRITE TOOLS
             │                   │
             ▼                   ▼
           Logs                Restart
          Metrics               Scale
           Status               Redrive
          Config             Change config
             │                   │
          Lower risk          Higher risk

This led to an important design principle:

Start with read-only autonomy. Earn write autonomy through evidence.

A real example: DLQ remediation

Consider a microservices application consuming messages from SQS.

A DLQ starts accumulating messages.

An agent might discover:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
DLQ messages increasing
        │
        ▼
Consumer errors detected
        │
        ▼
Application deployment introduced
        │
        ▼
Messages failing validation

The agent might recommend:

Do not redrive the messages yet. The consumer appears to have a deterministic application error.

That is a much better outcome than blindly redriving the DLQ.

Because otherwise:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
DLQ
 │
 ▼
Redrive
 │
 ▼
Consumer
 │
 ▼
Same bug
 │
 ▼
DLQ again

We have automated a loop rather than fixing the problem.

This is exactly why autonomous remediation needs context and verification.

Controlled remediation

Rather than immediately allowing:

1
Agent → AWS API → Production

I prefer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
                 Agent
                   │
                   ▼
              Investigation
                   │
                   ▼
              Recommendation
                   │
                   ▼
             Risk Assessment
                   │
                   ▼
             Human Approval
                   │
                   ▼
              Remediation
                   │
                   ▼
              Verification

For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
┌─────────────────────────────────────┐
│ Incident Investigation              │
│                                     │
│ Likely cause: payment-service       │
│ connection exhaustion               │
│                                     │
│ Confidence: High                    │
│                                     │
│ Suggested remediation:              │
│ Restart 3 unhealthy ECS tasks       │
│                                     │
│ Blast radius: 3 tasks               │
│                                     │
│ [ Approve ]          [ Reject ]      │
└─────────────────────────────────────┘

The agent becomes an experienced engineer’s assistant rather than an uncontrolled automation engine.

Not every remediation has the same risk

I would classify remediation actions by blast radius.

🟢 Low risk

Examples:

collect additional logs
retry a diagnostic query
retrieve service status
inspect deployment history
restart a single demonstrably unhealthy task

🟡 Medium risk

Examples:

restart multiple tasks
scale a service
redrive messages
disable an unhealthy target

🔴 High risk

Examples:

modify infrastructure
change security configuration
modify IAM
change database configuration
delete resources
modify production deployment configuration

The agent may be able to recommend all three categories.

That doesn’t mean it should be allowed to execute all three.

The principle I want to follow

I see agent autonomy as a progression:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Level 0
Observe
   │
   ▼
Level 1
Investigate
   │
   ▼
Level 2
Recommend
   │
   ▼
Level 3
Request approval
   │
   ▼
Level 4
Execute controlled remediation
   │
   ▼
Level 5
Verify outcome
   │
   ▼
Level 6
Learn from result

This is a much safer way to introduce agentic automation into production systems.

Don’t start with:

“Give the AI production access.”

Start with:

“Give the AI read access and prove that its investigations are useful.”

Then progressively increase autonomy.

Remediation isn’t complete until we verify it

Another important realization was that executing an action isn’t the end of the workflow.

Suppose the agent restarts an ECS task.

A bad workflow would be:

1
2
3
Restart task
    ↓
Done

A better workflow is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
Identify problem
      ↓
Recommend remediation
      ↓
Approval
      ↓
Execute
      ↓
Wait
      ↓
Check service health
      ↓
Check error rate
      ↓
Check logs
      ↓
Did the problem disappear?
      │
      ├── Yes → Resolve
      │
      └── No → Escalate

This is a critical distinction.

Otherwise we’re automating actions rather than automating operations.

The MLOps problem begins here

Once the agent starts making operational recommendations, simply asking:

“Does the answer look good?”

isn’t enough.

I want to capture the complete decision chain:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
Incident
   ↓
Evidence collected
   ↓
Agent analysis
   ↓
Root-cause hypothesis
   ↓
Confidence
   ↓
Recommended action
   ↓
Human decision
   ↓
Actual remediation
   ↓
Outcome

For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
Incident #1042

Diagnosis:
Database connection exhaustion

Confidence:
High

Recommendation:
Restart 3 ECS tasks

Human:
Approved

Result:
Error rate returned to normal

Evaluation:
Successful remediation

But we also need negative examples:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
Incident #1057

Diagnosis:
Payment service timeout

Recommendation:
Redrive DLQ messages

Human:
Rejected

Actual root cause:
Application validation bug

Evaluation:
Incorrect remediation recommendation

This data is extremely valuable.

It becomes an evaluation dataset for the agent.

From AI demo to measurable engineering system

This is where I see the project evolving beyond a simple LLM integration.

The long-term loop becomes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
Incident
   │
   ▼
Agent Investigation
   │
   ▼
Recommendation
   │
   ▼
Human / Policy Decision
   │
   ▼
Remediation
   │
   ▼
Outcome
   │
   ▼
Evaluation Dataset
   │
   ▼
Agent / Prompt / Model Improvement

Now I can start measuring:

investigation accuracy
root-cause hypothesis accuracy
false positives
false negatives
recommendation quality
remediation success rate
time to first useful hypothesis
mean time to resolution
human approval rate
rollback rate
model latency
inference failures

This is where MLOps becomes relevant.

The model isn’t just producing text.

It is participating in an operational workflow whose outcomes can be measured.

Where I want to take the architecture

The current implementation is intentionally small.

But the architecture gives me a path to evolve it.

Current

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
                    CloudWatch
                         │
                         ▼
                  Read-only Tools
                         │
                         ▼
                   Strands Agent
                         │
                         ▼
                    Local Llama
                         │
                         ▼
                Investigation Report

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
                    CloudWatch
                         │
                         ▼
                  Read-only Tools
                         │
                         ▼
                   Strands Agent
                         │
                 ┌───────┴───────┐
                 ▼               ▼
           Investigation    Recommendation
                                 │
                                 ▼
                           Risk Assessment
                                 │
                                 ▼
                           Human Approval
                                 │
                                 ▼
                         Remediation Tools
                                 │
                                 ▼
                            Verification

Eventually

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
                         Agent
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
           Observe       Reason         Act
              │            │            │
              └────────────┼────────────┘
                           ▼
                        Verify
                           │
                           ▼
                         Learn

Adding memory and RAG

Another limitation of the first version is that the agent only knows what it can observe during the current investigation.

But experienced engineers don’t work that way.

They remember:

“We’ve seen this incident before.”

“This service has a history of connection pool issues.”

“The last time this happened, the deployment was rolled back.”

That leads naturally to the next evolution:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
Current Incident
       │
       ▼
Current Evidence
       │
       ▼
      RAG
       │
       ├── Previous incidents
       ├── Runbooks
       ├── Architecture docs
       ├── Service documentation
       └── Historical remediation
       │
       ▼
Better Investigation

Eventually, memory can capture the operational history of the system.

The agent can then move from:

“What is happening?”

to:

“What is happening, have we seen it before, and what worked last time?”

Why the local model mattered

It would be easy to focus on the fact that I hosted Llama 3.1 8B on EC2.

But that isn’t really the main point.

The model choice forced me to think about the entire inference stack.

With a managed model:

1
2
3
4
5
6
7
Application
    │
    ▼
API
    │
    ▼
Model

A significant amount of infrastructure is abstracted away.

With a self-hosted model:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Application
    │
    ▼
Network
    │
    ▼
EC2
    │
    ▼
Ollama
    │
    ▼
Llama 3.1 8B

Now I have to consider:

instance sizing
memory
inference latency
model loading
concurrency
networking
security groups
authentication
availability
model versioning
health checks
monitoring
compute cost

That was exactly the learning opportunity I wanted.

The experiment changed from:

“How do I call an LLM?”

to:

“How do I operate an LLM as part of a production-oriented engineering system?”

The bigger architecture

Putting everything together, the direction looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
                         Microservices
                              │
                              ▼
                       Observability
                              │
                    ┌─────────┴─────────┐
                    ▼                   ▼
                 Logs                Metrics
                    │                   │
                    └─────────┬─────────┘
                              ▼
                        Agent Tools
                              │
                              ▼
                       Strands Agent
                              │
             ┌────────────────┼────────────────┐
             ▼                ▼                ▼
       Investigation        RAG             Memory
             │                │                │
             └────────────────┼────────────────┘
                              ▼
                       Local Llama 3.1
                           on EC2
                              │
                              ▼
                       Recommendation
                              │
                              ▼
                        Risk Engine
                              │
                   ┌──────────┴──────────┐
                   ▼                     ▼
             Auto-safe              Human Approval
              Actions                    │
                   │                     ▼
                   │                Remediation
                   │                     │
                   └──────────┬──────────┘
                              ▼
                          Verification
                              │
                              ▼
                           Evaluation
                              │
                              ▼
                       Model / Agent
                         Improvement

The interesting part isn’t any individual component.

It’s the progression:

Observability → Evidence → Agent → Reasoning → Recommendation → Controlled Action → Verification → Learning

What I learned

There are several lessons I took away from this implementation.

1. An agent is not just an LLM with a prompt

The useful abstraction is:

1
2
3
4
5
6
7
Model
+
Tools
+
Context
+
Decision loop

The model becomes one component of the system.

2. Good tools matter more than clever prompts

If the agent has poor access to operational data, a better prompt won’t solve the problem.

If the tools provide:

accurate logs
useful metrics
service context
deployment information
structured evidence

the model has a much better foundation for reasoning.

3. Deterministic work should remain deterministic

Calculations, filtering, aggregation and validation don’t need an LLM.

Let Python do them.

Let the model focus on:

reasoning
correlation
summarization
hypothesis generation
explanation

That makes the overall system easier to control.

4. Local models are an architectural choice

Running the model myself wasn’t simply about avoiding API costs.

It gave me ownership of:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
Model
 ↓
Serving
 ↓
Infrastructure
 ↓
Networking
 ↓
Security
 ↓
Observability
 ↓
Evaluation
 ↓
Operations

That is a much broader engineering problem.

And that is exactly why I wanted to do it.

5. Autonomous remediation should be earned

The most important lesson may be this one.

An agent that can read production data is useful.

An agent that can modify production systems is powerful.

An agent that can modify production systems without appropriate controls is dangerous.

So I don’t see autonomy as a binary switch.

I see it as a progression:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
Observe
  ↓
Investigate
  ↓
Recommend
  ↓
Approve
  ↓
Execute
  ↓
Verify
  ↓
Learn

Result: From dashboards to controlled autonomy

This brings me back to the idea behind my original Dashboards to Decisions work.

The progression is now:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
                    Observability
                         │
                         ▼
                     Dashboard
                         │
                         ▼
                      Alert
                         │
                         ▼
                    Investigation
                         │
                         ▼
                       Agent
                         │
                         ▼
                  Decision Support
                         │
                         ▼
                   Recommendation
                         │
                         ▼
                  Controlled Action
                         │
                         ▼
                     Verification
                         │
                         ▼
                      Learning

A dashboard tells me:

Something is wrong.

An agent can help answer:

What appears to be happening, what evidence supports it, and what should I investigate next?

A mature agentic system could eventually go further:

Here is the recommended remediation, here is the blast radius, here is why I believe it is safe, and here is what I will verify afterward.

That is a much more interesting direction than simply putting a chatbot on top of CloudWatch.

Closing: The model is not the product

The most important takeaway from this project isn’t:

“I hosted Llama 3.1 8B on EC2.”

That was one implementation decision.

The bigger lesson is that the interesting engineering problem sits around the model.

I already had:

Microservices → Observability → Dashboards → Alerts

The next step was:

Observability → Evidence → Agent → Investigation → Decision Support

And the next evolution is:

Decision Support → Controlled Remediation → Verification → Learning

Hosting the model myself allowed me to explore much more of that stack instead of treating inference as a black box.

It also reinforced something I increasingly believe about Agentic AI in engineering:

The hard problem isn’t making an agent take an action.
The hard problem is deciding which actions it should be allowed to take.

A production agent needs more than intelligence.

It needs:

bounded tools
least-privilege permissions
guardrails
approval mechanisms
blast-radius controls
auditability
verification
evaluation
rollback strategies

That’s where Agentic AI starts becoming an engineering discipline, rather than simply an LLM integration.

And that is ultimately what I wanted to explore with this implementation.

The dashboard tells you where to look.
The agent helps you investigate why.
The engineer decides what should happen next.
And eventually, carefully controlled automation can take care of the safe parts.

That, for me, is the next step in going from dashboards to decisions — and from decisions to controlled autonomy.

From Dashboards to Decisions: Building an AI-Powered Microservices Troubleshooting System#

Using Strands Agents, CloudWatch and a locally hosted Llama 3.1 8B model on Amazon EC2#

Situation: We already had observability#

Task: Move from detection to investigation#

Action: Introduce an agent into the existing architecture#

The model didn’t need to be a paid API#

Why I didn’t simply send all the logs to the LLM#

Tools are the bridge between the agent and reality#

Deterministic processing before AI reasoning#

From errors to causal chains#

The output should be useful to an engineer#

What happened?#

What evidence supports it?#

What is the likely cause?#

What is uncertain?#

What should I investigate next?#

The first result: reducing the investigation loop#

Before#

After#

Then came the obvious question#

Investigation is not the same as remediation#

A real example: DLQ remediation#

Controlled remediation#

Not every remediation has the same risk#

🟢 Low risk#

🟡 Medium risk#

🔴 High risk#

The principle I want to follow#

Remediation isn’t complete until we verify it#

The MLOps problem begins here#

From AI demo to measurable engineering system#

Where I want to take the architecture#

Current#

Next#

Eventually#

Adding memory and RAG#

Why the local model mattered#

The bigger architecture#

What I learned#

1. An agent is not just an LLM with a prompt#

2. Good tools matter more than clever prompts#

3. Deterministic work should remain deterministic#

4. Local models are an architectural choice#

5. Autonomous remediation should be earned#

Result: From dashboards to controlled autonomy#

Closing: The model is not the product#

From Dashboards to Decisions: Building an AI-Powered Microservices Troubleshooting System

Using Strands Agents, CloudWatch and a locally hosted Llama 3.1 8B model on Amazon EC2

Situation: We already had observability

Task: Move from detection to investigation

Action: Introduce an agent into the existing architecture

The model didn’t need to be a paid API

Why I didn’t simply send all the logs to the LLM

Tools are the bridge between the agent and reality

Deterministic processing before AI reasoning

From errors to causal chains

The output should be useful to an engineer

What happened?

What evidence supports it?

What is the likely cause?

What is uncertain?

What should I investigate next?

The first result: reducing the investigation loop

Before

After

Then came the obvious question

Investigation is not the same as remediation

A real example: DLQ remediation

Controlled remediation

Not every remediation has the same risk

🟢 Low risk

🟡 Medium risk

🔴 High risk

The principle I want to follow

Remediation isn’t complete until we verify it

The MLOps problem begins here

From AI demo to measurable engineering system

Where I want to take the architecture

Current

Next

Eventually

Adding memory and RAG

Why the local model mattered

The bigger architecture

What I learned

1. An agent is not just an LLM with a prompt

2. Good tools matter more than clever prompts

3. Deterministic work should remain deterministic

4. Local models are an architectural choice

5. Autonomous remediation should be earned

Result: From dashboards to controlled autonomy

Closing: The model is not the product