Logging Microservice

The Logging Microservice is a crucial component the distributed application architecture of Navida Pro backend, designed to centralize and manage logging across multiple microservices. Its primary purpose is to collect, process, and store log messages generated by various microservices within the ecosystem. By consolidating logs in a centralized location, this microservice provides enhanced visibility, analysis, and troubleshooting capabilities for system administrators and developers.

Description:

In Navida pro backend system, where numerous microservices collaborate to deliver complex functionalities, effective logging becomes vital for monitoring, debugging, and maintaining the health of the entire system. The Logging Microservice offers a solution to the challenges posed by managing logs across these diverse microservices.

Key Features:

Log Aggregation:

The Logging Microservice acts as a central hub for collecting log messages generated by various microservices. It ensures that logs from different sources are efficiently gathered in one place.

Structured Logging:

To enhance the clarity and usability of logs, microservices produce structured log messages that include both the log content and the source service's identity. This enables efficient parsing and analysis of logs.

Flexible Logging Destinations:

The Logging Microservice supports versatile logging destinations, including log files and databases. Depending on the source of the log message, it can route the logs to the appropriate destination.

Dynamic Configuration:

The microservice incorporates a Config Server that facilitates dynamic configuration updates. This enables fine-tuning of logging behavior without requiring service restarts.

Efficient Processing:

A Logging Processor within the microservice analyzes incoming log messages, extracting source information and determining the appropriate destination for each log entry.

Granular Control:

Based on the source of log messages, the microservice routes logs to either a File Logger or a Database Logger, ensuring efficient management of log data.

Scalability:

The architecture can scale horizontally to accommodate growing log volumes and the addition of new microservices, ensuring the system remains performant and reliable.

Centralized Monitoring:

The consolidated logs in the Logging Microservice provide administrators and developers with a single point of access for monitoring system behavior, diagnosing issues, and proactively addressing concerns.

Class Diagrams

the kafkaproducer and kafkaConsumer should be implemented in a Library and should be used by all the services .

Sequence

How the log data is structured

The log producer services should send the topic to kafka in a prefined format. Ex.

{
  "source": "Microservice1",
  "message": "This is a log message from Microservice1",
  "uniqueId": "microservice1-unique-id"
}

Logging Microservice should handle errors gracefully and provide clear error messages in an audit file in cases where log messages cannot be processed or validated.

Logging Microservice should handle increasing log volumes gracefully and maintain consistent performance as the number of microservices grows

the service should be able to update the configuration of the Logging Microservice without requiring service restarts. This will allow us to fine-tune logging behavior based on changing needs.

This service will also offload some of the non transactional works from the mainstream services by converting them to an asynchronous pattern using kafka.