Skip to main content

E2E Debugging & Final Review - Slides

SLIDE DECK: MODULE 08 - E2E DEBUGGING & FINAL REVIEW - V3.0​

Total Duration: 480 minutes (180' Lecture + 120' Workshop + 180' Final Project) Audience: Fresher/Employee (Completed Modules 01-06 and Module 08: Security)

Slide 1: Title Page

  • Content:
    • (Company / Training Unit Logo)
    • MODULE 08: E2E DEBUGGING & FINAL REVIEW (Capstone Module)
    • "The Climax": From Coder to System Operator
    • (Synthesizing the full architecture: SAGA, Caching, Observability & Security)
    • Trainer: (Your Name)
    • Date: (Training Date)
  • Visualization: *
    • Key visual: A full orchestra (representing all microservices) playing in harmony. The "Conductor" (You/SRE) is looking at the "master score" (Observability Dashboards + Security Alerts).
  • Instructor Script:
    • "Welcome to Module 08, our 'capstone' module. In the previous modules, you learned to 'build' each instrument: the Order Service (SAGA), the Product Service (Caching), 'install' the 'sound system' (O11y), and 'lock the doors' (Security)."
    • "Today, we're not 'building.' We're going to learn to 'listen.' We will act as the 'Conductor' (SRE/Operator), learn to 'hear' when the orchestra is 'out of tune' (errors/slowdowns), and 'pinpoint' exactly which 'musician' (service) is playing incorrectly."
    • "This is the 'graduation' lesson that transforms you from a 'Coder' (builder) to an 'Operator' (maintainer)."

Slide 2: Session Agenda (Updated V3.0)

  • Content:
    • AGENDA (MODULE 08)
    • P1. Final Review: The Full Architecture (The "Final Boss")
      • (Synthesizing M01 - M06 and M08 Security)
    • P2. "The Climax": The 4-Step E2E Debugging (SRE Workflow)
      • (Alert -> Metrics -> Traces -> Logs)
    • P3. Workshop 07: "The Bug Hunt" (Live Debugging)
      • Scenario 1: Happy Path (Trace)
      • Scenario 2: Failure Path (SAGA + O11y)
      • Scenario 3: Security Breach (AuthN/AuthZ Debug) [NEW]
    • P4. Case Studies & Best Practices (SRE Culture)
      • (Error Budgets, Blameless Postmortems)
    • P5. Final Project Briefing
      • (Introducing the 5-part Rubric, including Security)
    • P6. Final Exam Preparation
    • P7. Q&A & Course Wrap-up
  • Visualization:
    • A 7-step timeline, designed as a "graduation path." P2 (SRE Workflow) and P3 (Workshop) are highlighted as the core. Scenario 3 (Security Breach) is added to P3.
  • Instructor Script:
    • "This is our final agenda."
    • "P1: We will 'review' the entire massive architecture we've built, including Module 08 Security."
    • "P2 and P3 are the 'heart' of this module: I will present the '4-Step Debugging Workflow' (now including Alerting), then I will 'live debug' (Workshop 07) 3 scenarios: 'Working,' 'Failed' (SAGA), and 'Blocked' (Security)."
    • "P4, we'll learn about SRE 'culture.' Finally, we'll brief the 'Final Project' (P5) with its 5-part rubric, and prepare for the 'Final Exam' (P6)."

Slide 3: Learning Objectives (Updated V3.0)

  • Content:
    • OBJECTIVES (AFTER THIS MODULE, YOU WILL BE ABLE TO...)
    • 1. Synthesize: Present the entire system architecture, explaining the links between Gateway, Services, MQ (SAGA), Caching, O11y, and Security. [Upgraded]
    • 2. Analyze: Execute the E2E debugging workflow (Alert -> Metrics -> Traces -> Logs) to isolate a failure path or bottleneck.
    • 3. Debug: Use a combination of Jaeger, Kibana, and Grafana to find the root cause of an E2E incident (including 401/403 Auth errors). [Upgraded]
    • 4. Prepare: Complete the "Final Project" (with the 5-part rubric) and be ready for the "Final Exam."
  • Visualization:
    • 4 icons: [Icon: System Diagram (Synthesize)], [Icon: Detective/Debug (Analyze)], [Icon: 3 Tools (Debug)], [Icon: Graduation Cap (Prepare)]. The "Diagram" and "Debug" icons are updated to include an [Icon: Shield] (Security).
  • Instructor Script:
    • "Today's objective is not to 'learn' new tools, but to 'synthesize' and 'analyze'."
    • "You must be able to 'present' the entire architecture (Objective 1), including the Security layers."
    • "Most importantly, you must be able to 'debug' (Objective 3) the system using all 3 tools, including debugging a '403 Forbidden' error from Kong."
    • "Completing this module means you are ready for the 'Final Project' and 'Final Exam'."

Slide 4: P1 - Final Review: The Full Architecture (The "Final Boss") (UPGRADED)

  • Content:
    • P1: FINAL ARCHITECTURE REVIEW
    • (Synthesizing all modules)
    • M01 (Foundation): User, Product Services (FastAPI).
    • M02 (Gateway): Kong (Routing, Auth, Rate Limit).
    • M03 (Async): RabbitMQ (Producer/Consumer).
    • M04 (Resilience): SAGA Pattern (Choreography, Compensation).
    • M06 (Performance): Redis Cache (Cache-Aside, Anti-Stampede).
    • M05 (Observability):
      • OTel/Jaeger (Tracing)
      • ELK Stack (Logging)
      • Prometheus/Grafana (Metrics & Alerting)
    • M08 (Security): [Upgraded]
      • Kong (JWT/ACL) (Perimeter)
      • mTLS / Internal JWT (S2S Trust)
      • Docker/K8s Secrets (Secrets)
      • Secure Logging (O11y Hygiene)
  • Visualization: *
    • A "massive" and "professional" architecture diagram, showing ALL components:
      • Client -> [Kong]
      • Kong -> [User Svc] -> [User DB] (with Redis Cache nearby)
      • Kong -> [Product Svc] -> [Product DB] (with Redis Cache nearby)
      • Kong -> [Order Svc] -> [Order DB] (connected to RabbitMQ)
      • [RabbitMQ] (as SAGA hub) connecting [Payment Svc] and [Inventory Svc].
      • ALL components (Services, Kong, RabbitMQ, Redis, DBs) are 'exporting' O11y data to:
        • -> [OTel Collector] -> [Jaeger]
        • -> [Filebeat] -> [ELK Stack]
        • <- (Pull) [Prometheus] -> [Grafana (Alerts)]
      • Security Overlays: An [Icon: Lock] on [Kong] (for JWT/ACL). Internal "S2S" arrows are in an [Icon: Tunnel] (for mTLS/JWT). An [Icon: Vault] [Secrets] points to Services. An [Icon: Filter/Mask] is on [ELK Stack] (for PII Masking).
  • Instructor Script:
    • "Here it is. [Point to Diagram] This is the 'final boss' we have built together over this entire course."
    • "It's not just 'Safe' (SAGA), 'Fast' (Cache), and 'Visible' (O11y); it is now also 'Secure'."
    • "[Point to new icons] The 'Gate' (Kong) is 'locked' (JWT/ACL). 'Internal' (S2S) traffic is 'trusted' (mTLS). 'Passwords' are 'vaulted' (Secrets). And 'Logs' are 'clean' (Masking)."
    • "Today, we will operate this 'hardened' machine."

Slide 5: P2 - "The Climax": The 4-Step E2E Debugging (SRE Workflow)

  • Content:
    • P2: THE "BUG HUNTING" PROCESS (SRE WORKFLOW)
    • "Alert -> Metrics -> Traces -> Logs"
    • Scenario: 3 AM, phone buzzes (Alert).
    • THE 4-STEP PROCESS:
    • STEP 1: ALERT (The What?)
      • [Point to Slack] Read Alert: HighErrorRate: 5xx > 5% on "payment-service".
      • Conclusion: "PAYMENT SERVICE IS ON FIRE."
    • STEP 2: METRICS (The Impact?)
      • [Point to Grafana] Open payment-service Dashboard.
      • Find: Error Rate (RED) 5%, p99 Latency (RED) spiked to 5 seconds.
      • Conclusion: "IT'S FAILING & SLOW AT PAYMENT."
    • STEP 3: TRACING (The Where? - Use Exemplar)
      • [Point to Grafana] Click the 'dot' (spike) on the 5s Latency graph -> Click [View Trace in Jaeger].
      • [Point to Jaeger] Open the 'waterfall'. See Span 'ValidateCreditCard' (4950ms, RED).
      • Conclusion: "FAILURE (AND SLOWNESS) IS AT THE 'VALIDATECREDITCARD' STEP."
    • STEP 4: LOGGING (The Why?)
      • [Point to Jaeger] Copy the trace_id from the failed Span.
      • [Point to Kibana] Paste trace_id into Kibana search bar.
      • Find: The JSON Log: {"level": "ERROR", "trace_id": "...", "message": "Third-party payment gateway (Stripe) API timeout"}
      • Conclusion: "IT FAILED BECAUSE THE STRIPE API TIMED OUT."
  • Visualization: *
    • A 4-step "flowchart" showing the V2.0 debugging process:
    • 1. [Slack Alert (Red)] (Alert) -> (Detect) ->
    • 2. [Grafana Dashboard (RED)] (Metrics) -> (Click 'Exemplar') ->
    • 3. [Jaeger UI (Red Waterfall)] (Tracing) -> (Copy 'trace_id') ->
    • 4. [Kibana UI (Error Log)] (Logging) -> (Find Root Cause).
  • Instructor Script:
    • "This is the 'golden' workflow that every real SRE/DevOps engineer uses, and it's the core of Workshop 07. 'Memorize' it."
    • "STEP 1: [Point to Slack] 'Alert' tells you 'WHAT' is broken. (Payment Service)."
    • "STEP 2: [Point to Grafana] 'Metrics' (Grafana) confirm the 'IMPACT.' (5% Error, 5s Latency)."
    • "STEP 3: [Point to Grafana/Jaeger] 'Tracing' (Jaeger) (via 'Exemplar' or search) tells you 'WHERE' it's broken. (The 'ValidateCreditCard' Span)."
    • "STEP 4: [Point to Kibana] 'Logging' (Kibana) (by searching the trace_id) tells you 'WHY' it's broken. (Stripe timeout)."
    • "Alert -> Metrics -> Traces -> Logs. 4 steps. 60 seconds. That is the goal."

Slide 6: Section Intro - Workshop 07

  • Content:
    • (Based on fsa-04-section-intro.html template)
    • 02. WORKSHOP 07: E2E DEBUGGING (LIVE)
    • "Fire Drill": Hunting Bugs in the System
  • Visualization:
    • (Based on template)
    • Background image (slide-04.jpg).
    • Title 02. WORKSHOP 07: E2E DEBUGGING (LIVE) (text-5xl).
    • Subtitle "Fire Drill": Hunting Bugs in the System (text-3xl).
  • Instructor Script:
    • "Enough theory. Now it's the 'Fire Drill.' I'm going to 'start a fire' (cause an error), and we will 'put it out' (debug) together."

Slide 7: P3 - Workshop 07: "The Bug Hunt" (Live Debugging) (UPGRADED)

  • Content:
    • WORKSHOP 07: "THE BUG HUNT" (LIVE)
    • Goal: Apply the 4-step workflow (Metrics -> Traces -> Logs)
    • Trainer-led Demo:
    • Scenario 1: "Happy Path" (User places order successfully)
      1. (Postman) Send 1 POST /orders request (Happy).
      2. (Grafana) Show Request Rate (RED) increase, Error Rate (RED) = 0.
      3. (Jaeger) Find the 1 SAGA trace (Order -> Payment -> Inventory), show it's all 'green'.
      4. (Kibana) Copy trace_id -> Search Kibana -> Show the JSON logs for all 3 services.
    • Scenario 2: "Failure Path" (SAGA fail)
      1. (Config) Turn on "Failure Switch" (ENV INVENTORY_FAIL_RATE=1.0).
      2. (Postman) Send 1 POST /orders request (Failure).
      3. THE HUNT: Use Metrics (Grafana) -> Traces (Jaeger) -> Logs (Kibana) to prove the SAGA 'rolled back' successfully.
    • Scenario 3: "Security Breach" (AuthZ fail) [NEW]
      1. (Config) Remove admin ACL from Order Service on Kong.
      2. (Postman) Send POST /orders with a JWT from user Bob (group user, no admin rights).
      3. THE HUNT:
      4. (Metrics) See 403 Forbidden Error Rate spike on Kong (or Order Service).
      5. (Tracing) Find the 'red' Trace, see it fails at the Gateway or Order Service.
      6. (Logging) Search trace_id -> Find log: User 'Bob' lacks required group 'admin'.
  • Visualization: *
    • 3 columns (Scenario 1, 2, 3).
    • Scenario 1 (Happy): An (green waterfall).
    • Scenario 2 (Failure): An (waterfall with a red Span and compensation Spans).
    • Scenario 3 (Security): An (Trace is 'red' at the very first Span Kong).
  • Instructor Script:
    • "This is Workshop 07. I will demo 3 scenarios."
    • "Scenario 1 (Happy Path): I'll place a successful order and 'prove' it by 'walking through' all 3 tools (Grafana -> Jaeger -> Kibana) to show it's 'green'."
    • "Scenario 2 (Failure Path): I'll 'turn on' the 'Failure Switch' in Inventory Service (simulating out of stock)."
    • "Then, I will perform the '4-step hunt' exactly like Slide 5: I'll start from 'Metrics' (see the error), 'jump' to 'Tracing' (see the SAGA rollback), and 'end' at 'Logging' (find the 'Out of stock' log)."
    • "Scenario 3 (Security): [NEW] I'll simulate a user Bob (no permissions) trying to 'place an order'. We'll use the '4-step hunt' to 'catch' the 403 Forbidden error, and see if it 'dies' at Kong or the Service."

Slide 8: P4 - Case Studies & Best Practices (SRE Culture) (UPGRADED)

  • Content:
    • P4: CASE STUDIES & BEST PRACTICES (SRE CULTURE)
    • Case Study 1: Google SRE & "Error Budgets"
      • Philosophy: Do not aim for 100% Uptime. 100% is impossible and expensive.
      • SLO (Target): Set a 99.9% Uptime target (e.g., 43 mins downtime/month).
      • Error Budget: You have 43 minutes to "be broken."
      • The Rule:
        • If "Error Budget" remains -> Dev team is "allowed" to "Deploy" (new features).
        • If "Error Budget" is spent -> Dev team is "FROZEN" (Code Freeze). No new features, only fix bugs until the SLO is restored.
    • Case Study 2: Netflix & "Automated Canary Analysis" (Kayenta)
      • Problem: Deploy a new service (v2) to 1% of users. How do you know if v2 is "better" or "worse" than v1?
      • Solution: Don't use human eyes.
      • Kayenta (Tool): Automatically 'pulls' Metrics (from Prometheus/Grafana) for v1 and v2.
      • Compares 'RED' (Rate, Errors, Duration) of v1 vs. v2.
      • Automatically decides:
        • (v2 > v1) -> Auto-'promote' v2 to 10%, 50%, 100%.
        • (v2 < v1) -> Auto-'rollback' v2 to 0%.
    • Best Practice: Blameless Postmortems
      • When an incident happens, the goal is not to find "Who broke it?"
      • The goal is to find "Why did the process allow this to happen?"
      • Example: "Why didn't the Alert fire? Why was the Runbook wrong? Why didn't Test catch this?"
      • (Includes "Blameless Security Postmortems" - Why did PII leak?)
  • Visualization: * *
    • Diagram 1 (Error Budget): A 'budget' bar of 43 mins. [Deploy] (green) -> 'Budget' decreases. [Code Freeze] (red) -> 'Budget' replenishes.
    • Diagram 2 (Kayenta): [v1 (99%)] and [v2 (1%)] -> (Metrics) -> [Kayenta (Compare)] -> (Decision) -> [Auto-Promote] or [Auto-Rollback].
  • Instructor Script:
    • "This part is about 'SRE culture,' which is what makes these big companies run."
    • "Case Study 1: Google SRE. They never target 100% uptime. They target 99.9%, which gives the team a '43-minute error budget' per month."
    • "[Point to Diagram 1] If the team 'spends' that 43 minutes (system is buggy), the Dev team is 'punished' (Code Freeze) - no new features, only bug fixes, until the system is stable. This 'balances' new 'Features' vs. 'Stability'."
    • "Case Study 2: Netflix. They 'automate' Observability. [Point to Diagram 2] When they deploy v2, their 'bot' (Kayenta) 'watches' the Metrics (RED) of v1 and v2. If v2 is 'worse' -> Auto-rollback. No human needed."
    • "Finally, Best Practice: 'Blameless Postmortems.' When the system crashes, we don't ask 'Who did it?' We ask 'Why did our process fail to catch this?' (This includes security breaches)."

Slide 9: Section Intro - Final Project & Exam

  • Content:
    • (Based on fsa-04-section-intro.html template)
    • 03. FINAL PROJECT & EXAM
    • Capstone Project & Final Exam Preparation
  • Visualization:
    • (Based on template)
    • Background image (slide-04.jpg).
    • Title 03. FINAL PROJECT & EXAM (text-5xl).
    • Subtitle Capstone Project & Final Exam Preparation (text-3xl).
  • Instructor Script:
    • "Final part. We will talk about the 'Final Project' and the 'Final Exam'."

Slide 10: P5 - Final Project Briefing (UPGRADED V3.0)

  • Content:
    • P5: FINAL PROJECT BRIEFING
    • Mission: "Complete the entire project."
    • Requirement: Integrate ALL modules (M01-M06 & M08).
    • GRADUATION RUBRIC (5 SECTIONS): [Upgraded]
    • A. Architecture & Operations (20%)
      • [ ] (M02) Kong Gateway works (Routing).
      • [ ] (M05) Runs on docker-compose (OTel Collector, ELK, Prometheus...).
    • B. Resilience (SAGA) (25%)
      • [ ] (M04) SAGA Choreography (Happy Path) succeeds.
      • [ ] (M04) SAGA Choreography (Failure Path) 'rolls back' (Compensation).
      • [ ] (M03) Uses durable, persistent, manual_ack, DLQ.
    • C. Performance (Caching) (20%)
      • [ ] (M06) Cache-Aside applied (with v1:key, TTL, Anti-Stampede).
      • [ ] (M06) Has 'Fallback' (no 500 error) when Redis is down.
    • D. Observability (O11y) (20%)
      • [ ] (M05) trace_id flows through all 3 services and RabbitMQ (Jaeger).
      • [ ] (M05) Logs are JSON + trace_id (Kibana).
      • [ ] (M05) Has a RED Dashboard (Grafana) + Alert Rule (PromQL).
    • E. Security (15%) [NEW]
      • [ ] (M08) Kong 'locks' a service with JWT Plugin.
      • [ ] (M08) (Bonus) Authorization with ACL Plugin.
      • [ ] (M08) NO hard-coded passwords (uses Docker Secrets).
      • [ ] (M08) NO Tokens/PII in logs (Kibana).
      • [ ] (M08) Redis/RabbitMQ not using default passwords.
  • Visualization: *
    • A "massive" checklist divided into 5 sections (A, B, C, D, E), with the new "E. Security" section added.
  • Instructor Script:
    • "This is the 'Final Project.' The mission is 'complete the entire project.'"
    • "[Point to Rubric] This is the 5-part 'Grading Rubric.' Section A (Operations), B (SAGA), C (Cache), D (O11y) are what we've done."
    • "[EMPHASIZE] Section E (Security) is 'new,' based on Module 08. I will 'check' 5 things: Is Kong 'locked' (JWT/ACL)? Are 'secrets' hidden (Docker Secrets)? Are 'logs' clean (no tokens)? And are Redis/RabbitMQ 'hardened'?"
    • "You must 'pass' all 5 sections."

Slide 11: P6 - Final Exam Preparation

  • Content:
    • P6: FINAL EXAM PREPARATION
    • Format:
      • Multiple Choice
      • Short Essay
      • Scenario Analysis
    • Duration: 240 minutes (4 hours).
    • Key Topics (Includes M08): [Upgraded]
      • M01-M02: Bounded Context, Kong (Routing).
      • M03-M04: Sync vs. Async (When to use?), durable/manual_ack, SAGA (Choreography, Compensation).
      • M06: Cache-Aside (Flowchart), Invalidation (3 strategies), Anti-Stampede (Mutex).
      • M05: 3 Pillars, E2E Debug Workflow (Alert -> Metrics -> Traces -> Logs), trace_id, Structured Logging, ILM, Cardinality, Alerting.
      • M08: Gateway AuthN/AuthZ (JWT/ACL), S2S Trust (mTLS/Internal JWT), Safe Token Propagation, Secrets (Docker/K8s), Secure Logging (PII Masking).
  • Visualization: *
    • A "mind map" summarizing the 7 key modules and the most important "keywords" from each (now including M08).
  • Instructor Script:
    • "Finally, the 'Final Exam.' You will have 4 hours."
    • "It won't just be 'multiple choice.' There will be 'short essays' and 'scenario analysis.' For example: 'I give you an error scenario, you must present the E2E Debugging workflow'."
    • "[Point to Mind Map] This is your 'study map.' Make sure you 'master' these 'keywords.' Especially the 'Hard Parts' (SAGA, Anti-Stampede, E2E Debug Workflow, and Security Patterns) as these are the 'score-differentiating' questions."

Slide 12: Q&A

  • Content:
    • Q & A
    • Questions about the Final Project & Final Exam
  • Visualization:
    • A clean, minimal slide. Just the large letters "Q&A".
  • Instructor Script:
    • "Thank you. We will have 10 final minutes for Q&A about the 'Final Project' and 'Final Exam'."

Slide 13: Thank You & Course Wrap-up

  • Content:
    • THANK YOU & CONGRATULATIONS!
    • You have completed the Advanced Microservices Architecture Program
    • (Your Contact Info: Email, LinkedIn, etc.)
    • "You are no longer just a Coder. You are an Architect, an Operator, and a Defender." [Upgraded]
  • Visualization: *

[Image of a graduation cap or certificate of completion]

* A formal slide. Can re-use the "orchestra" image (Slide 1), but now the "Conductor" (the student) is 'bowing' to the audience (success).
  • Instructor Script:
    • "Thank you, and congratulations to all of you. You have completed a very long and technically 'heavy' journey."
    • "You went from 1 simple service (Module 1) to a system that is 'Safe' (Module 4), 'Fast' (Module 6), 'Visible' (Module 5), and 'Secure' (Module 8)."
    • "[UPGRADED] You are no longer just 'Coders' (who write code). You are 'Architects' (who design the system), 'Operators' (who keep it 'alive'), and 'Defenders' (who keep it 'safe')."
    • "Good luck on the Final Project and Final Exam. Thank you!"