Abstract

VibeBalancer is a distributed systems and backend engineering project focused on understanding concurrency, asynchronous programming, load balancing, fault tolerance, and monitoring in modern server architectures. The project was developed using Python and FastAPI to simulate real-world backend infrastructure concepts such as blocking vs non-blocking execution, concurrent request handling, server-side threading, custom load balancing strategies, backend fault recovery, and real-time metrics collection.

The system progressively evolved from a simple FastAPI server into a fault-tolerant distributed load balancing architecture capable of handling concurrent traffic efficiently while monitoring server health and system performance.

Links

Google Meet Link: meet.google.com/cif-srkq-dvh

Github Repository: https://github.com/codebybrijesh/VibeBalancer-Tasks

Aim

To design and implement a scalable backend infrastructure system that demonstrates:

• Concurrent request handling
• Blocking vs non-blocking behavior
• Asynchronous programming
• Custom load balancing algorithms
• Fault tolerance mechanisms
• Health monitoring and metrics collection

using Python and FastAPI.

Introduction

Modern distributed systems must efficiently handle thousands of concurrent users while maintaining reliability, scalability, and fault tolerance.

Backend servers need mechanisms to:

• Process multiple requests simultaneously
• Distribute traffic across servers
• Recover from failures
• Monitor infrastructure health in real time

VibeBalancer was developed to understand these real-world distributed system concepts through practical implementation.

The project started with learning GitHub workflows and Python fundamentals, followed by building FastAPI-based servers and gradually introducing asynchronous request handling, threading, custom load balancers, fault tolerant architectures, and monitoring systems.

The project demonstrates how modern backend infrastructure works internally and how technologies like load balancers and concurrent systems improve scalability and responsiveness.

Literature Survey and Technologies Used

Literature Survey

Distributed Systems and Concurrency

Modern web applications require efficient concurrent request handling to avoid performance bottlenecks. Blocking systems process requests sequentially, reducing scalability, whereas asynchronous and concurrent architectures improve throughput significantly.

Load Balancing

Load balancers distribute traffic across multiple backend servers to improve performance and reliability.

• Round Robin
• Least Connections

Fault Tolerance

Production-grade systems must detect failed servers and reroute traffic automatically to healthy instances. Health checks and retry mechanisms ensure high availability.

Monitoring and Observability

Monitoring systems collect:

• Application metrics
• System resource utilization
• Error statistics
• Request activity

to maintain infrastructure reliability and performance.

Technologies Used

Methodology

Phase 1 – GitHub Learning and Version Control

Implementation

• Learned Git and GitHub fundamentals
• Practiced repository creation and management
• Performed push and pull operations
• Used Git Bash for version control operations

Commands Practiced

git init
git clone
git add .
git commit -m "message"
git push
git pull
git status

Outcome

This phase established understanding of collaborative development workflows and source code management.

Phase 2 – Python Fundamentals

Implementation

• Variables and data types
• Loops
• Conditionals
• Functions
• Lists and dictionaries
• Basic problem solving

Outcome

Developed a strong foundation in Python programming and backend development concepts.

Phase 3 – FastAPI Server Development

Implementation

A FastAPI server was created with multiple API endpoints:

• /
• /work
• /info

The /work endpoint simulated server-side processing delays using asynchronous execution.

Example

await asyncio.sleep(delay)

Outcome

Learned backend API development and asynchronous request handling.

Phase 4 – Blocking vs Non-Blocking Behavior and Threading

Blocking Server

time.sleep(delay)

Async Server

await asyncio.sleep(delay)

Observations

• Blocking execution processed requests sequentially
• Sequential execution of 5 requests took approximately 15 seconds
• Threaded concurrent execution reduced response time significantly
• Asynchronous servers handled overlapping requests efficiently

Outcome

Understood concurrency, threading, and asynchronous programming in backend systems.

Phase 5 – Load Balancer Implementation

Algorithms Implemented

• Round Robin
• Least Connections

Features

• Request forwarding
• Connection tracking
• Concurrent request distribution

Outcome

Learned traffic distribution strategies and scalable backend architecture design.

Phase 6 – Fault Tolerant Load Balancer

Features Added

• Health check endpoints
• Automatic unhealthy server detection
• Retry mechanisms
• Recovery detection
• Graceful failure handling

Health Monitoring

GET /health

Failure Response

503 Service Unavailable

Outcome

Understood production-grade fault tolerant distributed systems.

Phase 7 – Monitoring and Metrics Collection

Metrics Endpoint

• Total requests
• Active connections
• Total errors
• CPU usage
• Memory usage

Monitoring Library

psutil

Metrics Example

{
  "total_requests": 10,
  "active_connections": 0,
  "total_errors": 0
}

Outcome

Learned real-time infrastructure monitoring and observability systems.

Results

• Concurrent request handling
• Blocking vs non-blocking execution
• Thread-based concurrency
• Asynchronous backend processing
• Custom load balancing
• Round Robin and Least Connections algorithms
• Fault tolerance mechanisms
• Health monitoring systems
• Metrics and observability

The final architecture efficiently handled concurrent requests while maintaining reliability and scalability.

Conclusion

VibeBalancer successfully implemented a distributed backend infrastructure system using FastAPI and Python.

The project provided practical exposure to:

• Backend engineering
• Distributed systems
• Concurrency
• Asynchronous programming
• Load balancing
• Infrastructure monitoring

The project simulated many concepts used in real-world production systems and significantly improved understanding of scalable backend architectures.

Future Scope

• Docker-based container deployment
• Kubernetes orchestration
• Reverse proxy integration using Nginx
• Auto-scaling systems
• Prometheus and Grafana dashboards
• Distributed tracing
• Rate limiting
• Circuit breaker implementation
• Cloud deployment
• SSL and authentication integration

References

1. FastAPI Documentation
2. Python Documentation
3. AsyncIO Documentation
4. HTTPX Documentation
5. psutil Documentation
6. Git Documentation
7. GitHub Documentation

Mentors and Mentees

Mentors

• Aadharsh Venkat
• Anirudh Nayak

Mentees

• Brijesh Sharma
• Advita Tantry
• Rahul Dhaker
• Aflah

IEEE Envision Project Report – VibeBalancer