Communication in Microservices
In a microservices architecture, an application is split into many small, independent services. To do anything useful, such as placing an order, those services must talk to each other. How they communicate affects the speed, reliability and flexibility of the whole system.
This guide explains the two main communication styles, synchronous and asynchronous, with Java and Spring Boot examples, a complete e-commerce order flow, and practical advice on choosing protocols, handling failures and keeping data consistent.
1. What Is Microservices Communication?
Microservices communication is the set of mechanisms and protocols that independent services use to exchange information over a network. Each microservice is a self-contained unit with its own logic and its own database, so services must collaborate to complete business processes.
Think of it as an orchestra: each musician (microservice) plays their own part, and only by working together do they produce the symphony (the application).
Communication is not just a feature; it is the lifeline of a microservices architecture. Without it, you would have a collection of isolated programs unable to achieve anything together.
2. Why Communication Is Harder in Microservices
In a monolith, one part of the code calls another with a simple method call in the same process. In microservices, that same call travels over the network, which changes everything:
| Method call in a monolith | Network call between microservices | |
|---|---|---|
| Speed | Nanoseconds | Milliseconds, often 100,000 times slower |
| Can it fail? | Practically never | Yes: timeouts, dropped connections, crashed services |
| Data | Objects passed directly | Data must be serialized, for example to JSON, and parsed again |
| Transactions | One database transaction covers everything | No shared transaction; each service has its own database |
| Changes | The compiler checks both sides | Both sides must agree on an API contract that can drift apart |
| Debugging | One stack trace | A request spans many services and logs |
Good communication design is about accepting these facts and planning for them. The Communication Challenges in Microservices guide looks at each problem in detail.
3. The Two Primary Styles of Communication
Microservices communicate in two main ways, each with its own uses and trade-offs.
| Synchronous (request–response) | Asynchronous (event-driven) | |
|---|---|---|
| How it works | The caller sends a request and waits for the answer | The sender publishes a message and carries on without waiting |
| Analogy | A phone call | A text message or a letter |
| Common technologies | HTTP/REST, gRPC, GraphQL | Apache Kafka, RabbitMQ, Amazon SQS and SNS |
| Coupling | Tight: both services must be running at the same time | Loose: the receiver can be offline and catch up later |
| Best for | Answers needed immediately, such as showing a page or checking stock | Background work and informing other services that something happened |
| Main drawback | Chains of calls: one slow or failed service affects all callers | More moving parts and harder end-to-end tracking |
3.1 Synchronous communication (request–response)
A client service sends a request and then waits for a response before it continues. It is a real-time “ask and wait for an answer” pattern.
- How it works: the caller blocks, or waits, until it receives a response or the request times out.
- Common protocols: HTTP/REST is the most widely used; gRPC is popular for its high performance.
- Best for: situations where the result is needed right away, such as a user waiting for a page to load.
- Drawback: it creates chains of dependencies. If one service is slow or down, the failure can cascade and make the whole request slow or unresponsive.
3.2 Asynchronous communication (event-driven)
A service (the publisher) sends a message but does not wait for a response. It announces that “something has happened”. Interested services (subscribers) receive the message and act on it in their own time.
- How it works: a message broker, such as RabbitMQ, Apache Kafka or Amazon SQS, sits in the middle. The publisher sends a message to the broker, and the broker delivers it to the subscribers.
- Common patterns: message queues and publish/subscribe (pub/sub).
- Best for: background processing, tasks that do not need an instant result, and decoupling services to improve resilience.
- Drawback: the system has more parts, and following a request from start to end takes extra tooling.
4. Synchronous Communication with REST and gRPC
4.1 Calling a REST API with RestClient
RestClient, available since Spring Framework 6.1 (Spring Boot 3.2), is the modern way to make synchronous HTTP calls in Spring. Always set timeouts, so a slow service cannot block your threads forever:
package com.example.order.client;
import java.time.Duration;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestClient;
@Component
public class InventoryClient {
private final RestClient restClient;
public InventoryClient(RestClient.Builder builder) {
SimpleClientHttpRequestFactory factory =
new SimpleClientHttpRequestFactory();
factory.setConnectTimeout(Duration.ofSeconds(2));
factory.setReadTimeout(Duration.ofSeconds(3));
this.restClient = builder
.baseUrl("http://localhost:8082")
.requestFactory(factory)
.build();
}
public boolean reserve(String productId, int quantity) {
ReservationResponse response = restClient.post()
.uri("/inventory/reservations")
.body(new ReservationRequest(productId, quantity))
.retrieve()
.body(ReservationResponse.class);
return response != null && response.reserved();
}
public record ReservationRequest(String productId, int quantity) {
}
public record ReservationResponse(boolean reserved) {
}
}
4.2 Declarative HTTP clients
Instead of writing each request by hand, you can describe the remote API as a Java interface and let Spring generate the implementation:
package com.example.order.client;
import java.math.BigDecimal;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.service.annotation.HttpExchange;
import org.springframework.web.service.annotation.PostExchange;
@HttpExchange("/payments")
public interface PaymentClient {
@PostExchange
PaymentResponse charge(@RequestBody PaymentRequest request);
record PaymentRequest(String orderId, String customerId, BigDecimal amount) {
}
record PaymentResponse(String transactionId, String status) {
}
}
package com.example.order.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestClient;
import org.springframework.web.client.support.RestClientAdapter;
import org.springframework.web.service.invoker.HttpServiceProxyFactory;
import com.example.order.client.PaymentClient;
@Configuration
public class ClientConfig {
@Bean
public PaymentClient paymentClient(RestClient.Builder builder) {
RestClient restClient = builder.baseUrl("http://localhost:8083").build();
return HttpServiceProxyFactory
.builderFor(RestClientAdapter.create(restClient))
.build()
.createClient(PaymentClient.class);
}
}
Now you simply call paymentClient.charge(request). Spring Cloud OpenFeign offers a similar style and is still common in existing projects.
4.3 gRPC for fast internal calls
gRPC uses HTTP/2 and a compact binary format called Protocol Buffers. You describe the service in a .proto file, and tools generate client and server code in many languages:
syntax = "proto3";
service InventoryService {
rpc CheckStock (StockRequest) returns (StockReply);
}
message StockRequest {
string product_id = 1;
int32 quantity = 2;
}
message StockReply {
bool available = 1;
}
gRPC is typically faster and smaller on the wire than REST with JSON, and it supports streaming. It is less convenient for browsers and for debugging with simple tools, so many teams use REST for public APIs and gRPC between internal services. Read more in Synchronous Communication in Microservices.
5. Asynchronous Communication with Message Brokers
5.1 Queues and topics
| Message queue (point-to-point) | Topic (publish/subscribe) | |
|---|---|---|
| Who receives a message? | One consumer | Every subscribed service |
| Typical use | Sharing work, such as “resize this image” | Announcing events, such as “order confirmed” |
| Analogy | A ticket queue at a bank counter | A radio broadcast |
5.2 Publishing an event with Apache Kafka
An event describes something that has already happened. Name it in the past tense, and include the data subscribers will need:
package com.example.events;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.List;
public record OrderConfirmedEvent(
String eventId,
String orderId,
String customerId,
List<OrderLine> lines,
BigDecimal total,
Instant confirmedAt) {
public record OrderLine(String productId, int quantity) {
}
}
package com.example.order.messaging;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;
import com.example.events.OrderConfirmedEvent;
@Component
public class OrderEventPublisher {
private static final String TOPIC = "order-confirmed";
private final KafkaTemplate<String, OrderConfirmedEvent> kafkaTemplate;
public OrderEventPublisher(KafkaTemplate<String, OrderConfirmedEvent> kafkaTemplate) {
this.kafkaTemplate = kafkaTemplate;
}
public void publish(OrderConfirmedEvent event) {
// The order ID is the key, so all events for one order keep their order
kafkaTemplate.send(TOPIC, event.orderId(), event);
}
}
spring:
kafka:
bootstrap-servers: localhost:9092
producer:
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
5.3 Subscribing to the event
package com.example.shipping.messaging;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
import com.example.events.OrderConfirmedEvent;
import com.example.shipping.ShippingService;
@Component
public class ShippingEventListener {
private final ShippingService shippingService;
public ShippingEventListener(ShippingService shippingService) {
this.shippingService = shippingService;
}
@KafkaListener(topics = "order-confirmed", groupId = "shipping-service")
public void onOrderConfirmed(OrderConfirmedEvent event) {
shippingService.createShipment(event.orderId(), event.lines());
}
}
spring:
kafka:
bootstrap-servers: localhost:9092
consumer:
auto-offset-reset: earliest
key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
value-deserializer: org.springframework.kafka.support.serializer.JsonDeserializer
properties:
spring.json.trusted.packages: "com.example.events"
The groupId matters. Every different group receives its own copy of each event, so the shipping service and the notification service (with groupId = "notification-service") both get every order. Several instances of the same group share the work instead, which is how a service scales out.
6. One-to-One and One-to-Many Interactions
Communication can also be described by how many receivers a message has:
| Pattern | Receivers | Style | Example |
|---|---|---|---|
| Request–response | One | Synchronous | Order service asks inventory: “Is product 123 in stock?” |
| Asynchronous request–response | One | Asynchronous | Send “generate invoice” and receive “invoice ready” later on a reply queue |
| One-way notification | One | Asynchronous | “Resize this uploaded image” put on a work queue |
| Publish/subscribe | Many | Asynchronous | “Order confirmed” received by shipping, notifications and analytics |
7. A Practical Example: The E-Commerce Order Flow
Let us walk through placing an order in an online store built with microservices.
Services involved:
- Order service: manages order creation and status.
- Inventory service: tracks product stock levels.
- Payment service: handles card payments.
- Shipping service: arranges delivery.
- Notification service: sends emails and alerts to the customer.
7.1 Step 1: the user clicks “Place Order” (synchronous)
- The front end sends a request to the order service to create a new order.
- The order service makes a synchronous call to the inventory service: “Are 2 units of product 123 available?” Answer: “Yes, they are reserved for you.”
- It then makes another synchronous call to the payment service: “Charge this customer $50.00.” Answer: “Payment successful. Transaction ID: TXN789.”
- Only after both succeed does the order service mark the order as confirmed and reply to the browser: “Your order was successful!”
7.2 Step 2: handling the aftermath (asynchronous)
- The order service publishes an
OrderConfirmedevent to the message broker. Its job is now done; it does not know or care who listens. - The shipping service receives the event, creates a shipping label and schedules a courier pickup.
- The notification service receives the same event and emails the customer an order summary.
Browser ---> Order Service
| (sync) ---> Inventory Service
| (sync) ---> Payment Service
|
+--- OrderConfirmed event ---> [ Broker ]
|
+-------------+-------------+
| |
Shipping Service Notification Service
7.3 The order service in code
package com.example.order;
import java.math.BigDecimal;
import java.time.Instant;
import java.util.UUID;
import org.springframework.stereotype.Service;
import com.example.events.OrderConfirmedEvent;
import com.example.order.client.InventoryClient;
import com.example.order.client.PaymentClient;
import com.example.order.client.PaymentClient.PaymentRequest;
import com.example.order.client.PaymentClient.PaymentResponse;
import com.example.order.messaging.OrderEventPublisher;
@Service
public class OrderService {
private final InventoryClient inventoryClient;
private final PaymentClient paymentClient;
private final OrderRepository orderRepository;
private final OrderEventPublisher eventPublisher;
public OrderService(InventoryClient inventoryClient, PaymentClient paymentClient,
OrderRepository orderRepository, OrderEventPublisher eventPublisher) {
this.inventoryClient = inventoryClient;
this.paymentClient = paymentClient;
this.orderRepository = orderRepository;
this.eventPublisher = eventPublisher;
}
public Order placeOrder(PlaceOrderRequest request) {
// Step 1a: synchronous - we need the answer before we can continue
if (!inventoryClient.reserve(request.productId(), request.quantity())) {
throw new OutOfStockException(request.productId());
}
// Step 1b: synchronous - the customer is waiting for the payment result
Order order = orderRepository.save(Order.pending(request));
BigDecimal amount = order.total();
PaymentResponse payment = paymentClient.charge(
new PaymentRequest(order.id(), request.customerId(), amount));
order = orderRepository.save(order.confirm(payment.transactionId()));
// Step 2: asynchronous - shipping and notifications happen in the background
eventPublisher.publish(new OrderConfirmedEvent(
UUID.randomUUID().toString(),
order.id(),
order.customerId(),
order.lines(),
order.total(),
Instant.now()));
return order;
}
}
This hybrid approach is very common. Synchronous calls are used where an immediate answer is essential, and asynchronous events decouple the follow-up steps. If the shipping service is temporarily down, the OrderConfirmed event simply waits in the broker until it comes back, so a shipping problem never stops customers from placing orders.
8. How to Choose the Right Style
| Ask yourself | If yes, prefer |
|---|---|
| Does the caller need the answer to continue right now? | Synchronous |
| Is a user waiting on screen for this exact result? | Synchronous |
| Can the work happen a few seconds or minutes later? | Asynchronous |
| Do several services need to react to the same thing? | Asynchronous (publish/subscribe) |
| Must the work survive the receiver being down for a while? | Asynchronous |
| Is the work slow or heavy, such as reports, video or bulk email? | Asynchronous |
| Do you need to absorb sudden traffic spikes? | Asynchronous (the queue acts as a buffer) |
A good rule of thumb: query synchronously, react asynchronously. Ask for data you need now with a request, and announce changes with events.
9. Protocols and Data Formats Compared
| Protocol | Style | Data format | Strengths | Typical use |
|---|---|---|---|---|
| REST over HTTP | Sync | JSON | Simple, universal, easy to debug | Public APIs and most service-to-service calls |
| gRPC | Sync and streaming | Protocol Buffers (binary) | Fast, compact, strongly typed contracts | High-volume internal calls |
| GraphQL | Sync | JSON | Clients ask for exactly the fields they need | APIs for varied front ends |
| WebSocket | Two-way | Any | Server can push updates to clients | Live dashboards, chat |
| AMQP (RabbitMQ) | Async | Any | Flexible routing, per-message acknowledgment | Task queues and routing-heavy messaging |
| Apache Kafka | Async | Any (often JSON or Avro) | Very high throughput; events are stored and can be replayed | Event streaming and event-driven systems |
| Cloud queues (SQS, Pub/Sub) | Async | Any | Fully managed, no servers to run | Cloud-native background work |
10. How Services Find Each Other
In the cloud, services start, stop and move all the time, so hard-coded addresses quickly break. Two building blocks solve this:
- Service discovery: services register themselves, for example in Eureka, Consul or Kubernetes, and callers look them up by name, such as
inventory-service. - API gateway: a single entry point for external clients that routes each request to the right service.
With message brokers, services do not need each other’s addresses at all; they only need to know the broker and the topic or queue name.
11. Making Communication Reliable
Networks fail, so every remote call needs protection. The most important tools are:
| Technique | Protects against | Learn more |
|---|---|---|
| Timeouts | Waiting forever for a slow service | Timeout pattern |
| Retries with backoff | Short, temporary failures | Retry pattern |
| Circuit breaker | Hammering a service that is already failing | Circuit breaker pattern |
| Bulkhead | One slow dependency using up all threads | Bulkhead pattern |
| Rate limiting | Too many requests overloading a service | Rate limiting pattern |
| Idempotency | Duplicate requests or messages causing duplicate effects | Section 11.2 below |
11.1 Retry and circuit breaker with Resilience4j
package com.example.order.client;
import org.springframework.stereotype.Component;
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
import io.github.resilience4j.retry.annotation.Retry;
@Component
public class StockChecker {
private final InventoryApi inventoryApi;
public StockChecker(InventoryApi inventoryApi) {
this.inventoryApi = inventoryApi;
}
// Reading stock is safe to retry; it changes nothing
@Retry(name = "inventory")
@CircuitBreaker(name = "inventory", fallbackMethod = "stockUnknown")
public StockStatus check(String productId) {
return inventoryApi.getStock(productId);
}
// Used when the inventory service is down or the circuit is open
private StockStatus stockUnknown(String productId, Throwable ex) {
return StockStatus.unknown(productId);
}
}
resilience4j:
retry:
instances:
inventory:
max-attempts: 3
wait-duration: 500ms
circuitbreaker:
instances:
inventory:
sliding-window-size: 10
failure-rate-threshold: 50
wait-duration-in-open-state: 30s
These annotations need the resilience4j-spring-boot3 dependency (or Spring Cloud Circuit Breaker) together with Spring AOP.
11.2 Idempotent message handling
Most brokers guarantee at-least-once delivery: a message can arrive more than once, for example after a restart. Consumers must therefore be idempotent, meaning that handling the same message twice has the same effect as handling it once:
package com.example.notification.messaging;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;
import com.example.events.OrderConfirmedEvent;
import com.example.notification.EmailService;
import com.example.notification.ProcessedEvent;
import com.example.notification.ProcessedEventRepository;
@Component
public class NotificationEventListener {
private final ProcessedEventRepository processedEvents;
private final EmailService emailService;
public NotificationEventListener(ProcessedEventRepository processedEvents, EmailService emailService) {
this.processedEvents = processedEvents;
this.emailService = emailService;
}
@KafkaListener(topics = "order-confirmed", groupId = "notification-service")
@Transactional
public void onOrderConfirmed(OrderConfirmedEvent event) {
// Skip events we have already handled
if (processedEvents.existsById(event.eventId())) {
return;
}
emailService.sendOrderConfirmation(event.customerId(), event.orderId());
processedEvents.save(new ProcessedEvent(event.eventId()));
}
}
12. Keeping Data Consistent Across Services
Each microservice owns its own database, so one database transaction cannot cover the whole order flow. Two patterns help:
- Saga pattern: a business process is split into local steps, each with a compensating action. If payment fails after stock was reserved, the saga releases the reserved stock.
- Transactional outbox: instead of saving the order and then publishing the event (two separate actions that can fail independently), the service saves the event into an outbox table in the same database transaction as the order. A separate process then publishes outbox rows to the broker. This avoids “order saved but event lost”.
The result is eventual consistency: all services agree after a short delay, rather than instantly. For most business processes, such as shipping after payment, this is perfectly acceptable. See Database per Service for why services keep separate data.
13. API Contracts and Versioning
Services evolve independently, so the contract between them, meaning the API or event format, must stay compatible:
- Add, do not remove or rename. New optional fields are safe; removing or renaming fields breaks consumers.
- Be tolerant when reading. Ignore unknown fields instead of failing, so producers can add fields freely.
- Version breaking changes, for example
/api/v2/ordersor a new event typeOrderConfirmedV2, and run old and new versions side by side for a while. - Document contracts with OpenAPI for REST,
.protofiles for gRPC and AsyncAPI or a schema registry for events. - Test contracts with consumer-driven contract tests, such as Spring Cloud Contract or Pact, so a provider change that breaks a consumer is caught before release.
14. Tracing Requests Across Services
When one user action touches five services, you need a way to follow it. Distributed tracing gives every request a trace ID that is passed along with each HTTP call and each message.
- With Spring Boot, add Micrometer Tracing with an OpenTelemetry or Brave bridge; trace IDs are then passed automatically through
RestClient,WebClientand Kafka. - Trace IDs appear in your log lines, so you can search all services’ logs for one request.
- Tools such as Zipkin, Jaeger or Grafana Tempo show a timeline of every call, which makes slow steps easy to spot.
15. Best Practices
- Prefer asynchronous events for anything that does not need an immediate answer.
- Keep synchronous call chains short; avoid one request triggering a long chain of calls.
- Set a timeout on every remote call, and add retries and circuit breakers where they make sense.
- Retry only safe operations, or use idempotency keys for operations that change data.
- Make message consumers idempotent, because messages can arrive more than once.
- Design events carefully: past-tense names, a unique event ID, and the data subscribers need.
- Use the outbox pattern when saving data and publishing an event must both happen.
- Keep contracts backward compatible, and version breaking changes.
- Use service discovery rather than fixed addresses.
- Add distributed tracing and correlation IDs from the start.
- Secure internal traffic too, with tokens or mutual TLS, not only the public edge.
16. Common Mistakes to Avoid
| Mistake | What goes wrong | Better approach |
|---|---|---|
| Making everything synchronous | A chain of calls is only as reliable as its weakest service | Use events for follow-up work |
| No timeouts | Threads pile up waiting, and the caller crashes too | Set connect and read timeouts on every client |
| Retrying payments blindly | Customers are charged twice | Use idempotency keys and retry only safe calls |
| Chatty services | One page needs dozens of calls, so it is slow | Return the data callers need in fewer calls, or aggregate |
| Sharing a database between services | Services become tightly coupled through tables | Each service owns its data and exposes it through APIs or events |
| Events that carry too little data | Every subscriber calls back to the publisher for details | Include the fields subscribers need |
| Ignoring duplicate messages | Duplicate emails, shipments or charges | Track processed event IDs |
| Breaking API changes without versions | Other services fail after a deployment | Add fields compatibly; version breaking changes |
17. Summary
Microservices communication is the nervous system that lets a distributed application work as one. Synchronous communication, using REST or gRPC, suits requests that need an immediate answer. Asynchronous communication, using brokers such as Kafka or RabbitMQ, decouples services and keeps the system working when parts fail. Most real systems combine both: query synchronously, react asynchronously. Protect every call with timeouts, retries and circuit breakers, make consumers idempotent, keep data consistent with sagas and the outbox pattern, and trace requests across services. With these practices, your services stay independent while working together reliably. Continue with Synchronous Communication and Asynchronous Communication for deeper dives.
18. Frequently Asked Questions (FAQ)
Q1. Is synchronous or asynchronous communication better?
Neither is better in general. They solve different problems:
- Use synchronous calls when the caller cannot continue without the answer, such as checking a price before showing it.
- Use asynchronous messages when the work can happen later or several services must react, such as sending emails after an order.
Most healthy systems use both: a short synchronous path for what the user waits on, and events for everything else.
Q2. Does asynchronous communication make my application slower?
Usually the opposite, from the user’s point of view. The user gets a reply as soon as the essential work is done, while slower tasks such as emails, reports and analytics run in the background.
The trade-off is that the follow-up work finishes a little later, typically milliseconds to seconds. That is fine for an email or a shipping label, but not for something the user needs to see immediately.
Q3. What is a message broker, and do I always need one?
A message broker is a server that receives messages from producers, stores them safely and delivers them to consumers. Examples are Apache Kafka, RabbitMQ and cloud services such as Amazon SQS.
You need one for reliable asynchronous communication: it keeps messages when consumers are offline and spreads work between instances. If all your communication is synchronous, you do not need a broker, but most microservice systems end up using one.
Q4. Kafka or RabbitMQ: which should I choose?
| Apache Kafka | RabbitMQ | |
|---|---|---|
| Model | Distributed log; events are kept and can be re-read | Traditional broker; messages are removed once processed |
| Strength | Very high throughput, replay, event streaming | Flexible routing, per-message acknowledgment, simple task queues |
| Good for | Event-driven systems, analytics, many subscribers | Work queues, request–reply over messages, complex routing |
Both are excellent. If you need to keep and replay event history, pick Kafka; for classic job queues with flexible routing, RabbitMQ is often simpler.
Q5. What is the difference between an event and a command?
- An event says that something has happened:
OrderConfirmed. The sender does not care who reacts, and any number of services may listen. - A command asks for something to be done:
ShipOrder. It is aimed at one specific receiver, which is expected to act.
Events keep services loosely coupled; commands make the dependency explicit. Name events in the past tense and commands as instructions.
Q6. Why should services not share one database to communicate?
Reading and writing each other’s tables seems easy, but it creates hidden coupling:
- One team cannot change its table structure without breaking other services.
- Services can no longer be deployed or scaled independently.
- Business rules are bypassed when another service writes data directly.
Each service should own its data and share it only through its API or events.
Q7. What is eventual consistency?
With eventual consistency, different services may briefly show different data, but they catch up shortly afterwards. For example, an order may be confirmed a second or two before the shipping service knows about it.
It is the normal result of asynchronous communication between services with separate databases. Design the user experience around it, for example by showing “Your order is being processed” instead of promising that everything is already done.
Q8. How do I handle a message that keeps failing?
Do not let one bad message block all the others. A common approach:
- Retry a few times with a short delay, in case the problem is temporary.
- If it still fails, move it to a dead-letter queue (or dead-letter topic).
- Alert the team, inspect the message, fix the cause and replay it if needed.
Spring for Apache Kafka supports this with a DefaultErrorHandler and a DeadLetterPublishingRecoverer; RabbitMQ has built-in dead-letter exchanges.
Q9. Are messages always delivered in order?
Not across the whole system. Typical guarantees:
- Kafka keeps order within one partition. Using the same key, such as the order ID, for related events puts them in the same partition, so they stay in order.
- RabbitMQ keeps order within a single queue with one consumer, but several consumers can process messages out of order.
If order matters, choose keys carefully, and make consumers able to cope with an occasional late or duplicate message, for example by checking version numbers.
Q10. What does idempotent mean, and why does it matter?
An operation is idempotent if doing it twice has the same effect as doing it once. “Set the order status to SHIPPED” is idempotent; “add $10 to the balance” is not.
It matters because networks and brokers can deliver the same request or message more than once. Make operations idempotent by:
- Recording processed message IDs and skipping duplicates.
- Sending an idempotency key with requests that change data, such as payments.
- Preferring “set” operations over “add” operations where possible.
Q11. Should I use REST or gRPC between my services?
Start with REST unless you have a reason not to: it is simple, works everywhere and is easy to test with a browser or curl.
Consider gRPC when:
- Services exchange a very large number of calls, and speed and payload size matter.
- You want strict, generated contracts across several programming languages.
- You need streaming in one or both directions.
Q12. How can one request avoid waiting on many services in a row?
Several techniques shorten long synchronous chains:
- Call in parallel when calls do not depend on each other.
- Keep a local copy of data you often need, updated through events, instead of asking each time.
- Move work to events if the user does not need to wait for it.
- Cache data that changes rarely.
As a guide, a user request should rarely need more than two or three synchronous hops.
Q13. What is the request–reply pattern over messaging?
It combines asynchronous transport with a response. The sender puts a request on a queue with a reply-to address and a correlation ID; the receiver processes it and sends the answer to the reply address with the same correlation ID.
It is useful for long-running work, such as generating a large report, where an HTTP call would time out. Spring’s ReplyingKafkaTemplate and RabbitMQ’s direct reply-to feature support it.
Q14. How do I secure communication between services?
- Encrypt traffic with TLS, even inside your network.
- Authenticate callers, for example by passing the user’s JWT along, using service-to-service tokens (OAuth 2 client credentials), or mutual TLS.
- Authorize each request in the receiving service.
- Secure the broker with authentication and topic-level permissions.
A service mesh such as Istio or Linkerd can add mutual TLS between services automatically.
Q15. What is a correlation ID?
A correlation ID is a unique value attached to a request when it enters the system and passed along with every call and message it causes. Every service writes it into its logs.
When something goes wrong, you search all logs for that one ID and see the complete story of the request across services. Distributed tracing tools use trace IDs in the same way, with timing information added.
Q16. How should I test communication between services?
- Unit tests: mock the client interfaces, such as
PaymentClient, to test your logic. - Integration tests: use WireMock to fake HTTP services, and Testcontainers to run a real Kafka or RabbitMQ in tests.
- Contract tests: tools such as Spring Cloud Contract or Pact check that providers still meet what consumers expect.
- End-to-end tests: a few tests of the most important journeys in a realistic environment.
Q17. Can a service be both a publisher and a subscriber?
Yes, and it is very common. The order service, for example, may publish OrderConfirmed and subscribe to PaymentFailed or ShipmentDelivered to update the order status.
Just avoid accidental loops, where service A reacts to B’s event by publishing an event that makes B publish again, and so on.
Q18. What is the difference between orchestration and choreography?
- Orchestration: one central service, the orchestrator, tells each service what to do and in which order, like a conductor.
- Choreography: there is no central controller; each service listens for events and decides on its own what to do next, like dancers reacting to each other.
Orchestration makes the flow easy to see and change in one place. Choreography keeps services more independent but makes the overall flow harder to follow. Many systems use orchestration for complex business processes and choreography for simple reactions.
Q19. How large should a message or event be?
Large enough that most subscribers do not need to call back for more details, but no larger. Good events usually contain:
- A unique event ID and a timestamp.
- The ID of the business object, such as the order ID.
- The key facts subscribers need, such as order lines and total.
Avoid putting files or images in messages; store them elsewhere and include a link. Brokers also have size limits, about 1 MB by default in Kafka.
Q20. When should I use WebSockets or server-sent events?
Use them when the server needs to push updates to the client as they happen:
- Live order tracking or delivery status.
- Chat and notifications.
- Dashboards with live numbers.
Server-sent events are simpler and enough when updates flow only from server to client. WebSockets allow two-way communication. Between backend services, message brokers are usually a better fit than either.