What is Service Discovery?
Service discovery is the process of finding the network location of a service so that a client can connect to it without relying on a permanently fixed address. A name, service type or query can return an address, port and other connection information. In dynamic systems, discovery updates that information as application instances start, stop or move.
The important distinction is that discovering an endpoint does not prove it is reachable, healthy or trustworthy. A printer can appear in a device list yet fail to print. An application can resolve a database name yet be blocked by a firewall. Service discovery answers where to connect; other parts of the system determine whether that connection works.
Why service discovery matters
A fixed IP address in a configuration file can work for a small, stable setup. It becomes fragile when containers are recreated, services scale across several machines or a failed instance is replaced. The logical name remains useful while the network endpoint changes.
Applications find databases, APIs and other dependencies by name.
Container platforms update endpoints as workloads change.
Devices advertise printers and other local services for users to select.
Gateways and proxies can use discovery information to refresh backend pools.
Operators can organise instances by environment, region, version or other supported metadata.
Discovery usually involves a control path that publishes or retrieves connection information, followed by a data path that carries application traffic. The directory does not necessarily sit in that traffic path. The infographic illustrates a registry based example; local multicast discovery uses a different arrangement.
How does service discovery work?
An illustrative web application needs an available instance of a service named inventory. Instead of storing one server’s address indefinitely, it uses a discovery mechanism to obtain usable endpoints.
For the registry illustration, imagine inventory has three private endpoints: 10.0.1.12:8080, 10.0.2.34:8080 and 10.0.3.56:8080. These addresses are examples, not live services. A client can select an eligible instance directly; a later health update can change which instances are offered.
Register or advertise: the service, an agent, a controller or an administrator publishes its location.
Maintain: the system updates that information when instances or their eligibility change.
Look up: the client or an intermediary queries a name or service catalogue.
Select: the client or routing component chooses a suitable endpoint.
Connect: the client sends application traffic through the configured network path.
Refresh: cached results and connections are updated when necessary; failed requests still need application handling.
These are logical stages, not six mandatory network transactions. An implementation can return extra records in one response, use cached information or push updates to subscribers. Some systems rely on configured registration rather than automatic scanning.
Consul’s service discovery explanation describes a catalogue that maps service identities to changing access information. A registry entry can include several instances of the same logical service.
Client side and server side discovery
Pattern | Who selects the backend? | Practical consequence |
|---|---|---|
Client side discovery | The client uses discovery information and chooses an endpoint. | Client libraries handle selection, refresh and failures. |
Server side discovery | A gateway, proxy or service networking layer chooses an endpoint. | Clients use a stable entry point; routing infrastructure handles backend selection. |
Local service browsing | A device learns advertised service instances and resolves a selected one. | Scope, permissions and multicast support affect what is visible. |
In client side discovery, the registry can be absent from the application traffic path after lookup. In server side discovery, traffic passes through the relevant routing layer or forwarding mechanism. The exact implementation may use a proxy, packet forwarding rules or another supported data plane.
Service discovery and load balancing are related but distinct. Discovery supplies endpoint information; balancing distributes traffic among eligible endpoints. A reverse proxy can consume discovery data, but a proxy with manually configured backends does not automatically provide dynamic discovery.
DNS discovery: more than an IP address
DNS can provide a familiar lookup interface for discovery. The records available and the client’s behaviour determine what information is used.
Record | Information supplied | Important distinction |
|---|---|---|
A / AAAA | IPv4 / IPv6 address for a name. | Does not specify an application port or prove health. |
SRV | Target hostname, port, priority and weight. | The client protocol must support the relevant SRV lookup. |
TXT | Additional descriptive or configuration data. | Meaning depends on the application or discovery convention. |
PTR in DNS-SD | Named instances of a requested service type. | Used for browsing before resolving a particular instance. |
RFC 2782, published in February 2000 on the Standards Track, defines SRV records. Lower numerical priority is preferred; weights influence selection among targets with the same priority. An SRV record is not an active health check, and an ordinary browser does not automatically use arbitrary SRV records to locate every website.
DNS based Service Discovery, or DNS-SD, is a specific convention defined in RFC 6763, published in February 2013 on the Standards Track. It combines service instance browsing with resolution through ordinary DNS record types. It can use conventional unicast DNS or multicast DNS; it is not synonymous with every DNS based cloud discovery system.
A practical DNS-SD example
Suppose a home office printer advertises an IPP printing service. A client browses the service type _ipp._tcp.local. A PTR answer identifies a named printer instance; SRV supplies its target hostname and port, TXT supplies service metadata, and address records provide the target’s IP information.
This is a conceptual sequence rather than a claim that every step needs a separate packet. The printer’s friendly service name can remain stable even when its address changes. Apple’s archived Bonjour operations guide explains publication, browsing and resolution using this model.
DNS-SD identifies services that are advertised. It is not a scan of every open port, and a device with no relevant advertisement may still be reachable by a known address.
mDNS, Bonjour and discovery across VLANs
Multicast DNS, or mDNS, provides DNS style queries on a local link without depending on a conventional DNS server for those queries. RFC 6762, published in February 2013 on the Standards Track, specifies UDP port 5353 and the multicast destinations 224.0.0.251 for IPv4 and ff02::fb for IPv6.
The .local. suffix has special mDNS semantics. Using it as an ordinary private unicast DNS suffix can cause confusing resolver behaviour. Multicast queries normally remain on their local link, so putting a printer and laptop into separate VLANs can remove automatic visibility even when some routed connections are allowed.
A deliberately configured discovery gateway or reflector may make selected advertisements visible across segments. The advertised service still needs an allowed application path: relaying discovery packets alone does not authorise printing, casting or file access. An IPv6 link local address is also scoped to its interface and cannot simply become a useful routed address through reflection.
Treat this as a network segmentation design question. Keep guest devices and untrusted equipment separated, then allow only the discovery and application flows you actually need. Some applications also require local network permission; Apple’s Bonjour guide explains that requirement alongside modern APIs.
Service discovery in Docker Compose
On the default Compose network, containers can discover other services by their Compose service names. If a web service and database share that network, the web application can connect to db on the database’s container port, for example db:5432. That differs from a port published on the host.
Docker’s Compose networking documentation explains that a replacement container can get a different IP address while retaining its service name. Existing connections can break, so the calling application must detect the failure, resolve again and reconnect.
The name is scoped to the relevant Docker network. It does not necessarily resolve from your laptop, a separate Compose project or the public internet. The default bridge outside Compose also differs from a user defined network. See the Docker guide for the wider container networking model.
Discovery is not readiness. A database name can resolve while the database is still starting or rejecting authentication. The application needs appropriate startup handling and retry behaviour; adding a DNS name does not solve dependency timing.
Service discovery in Kubernetes
In Kubernetes, a Service provides a stable abstraction for access to changing workloads. For a typical Service with a selector, the control plane tracks matching Pods through EndpointSlices. The network implementation uses eligible endpoints to carry Service traffic.
A normal Service’s DNS name usually resolves to its ClusterIP rather than directly listing every Pod address. A headless Service uses clusterIP: None and can expose endpoint addresses through DNS instead of allocating that virtual IP. The Kubernetes Service documentation explains this distinction.
For an illustrative Service named inventory in namespace lab, a cluster using the common cluster.local domain can provide inventory.lab.svc.cluster.local. A Pod in lab may resolve inventory using its DNS search configuration; a Pod elsewhere should qualify the namespace. The cluster domain is configurable, and these internal names are not automatically public internet names.
The Kubernetes DNS reference also describes SRV records for named Service ports. Do not equate this naming convention with local Bonjour browsing simply because both use DNS related mechanisms.
EndpointSlices record endpoint conditions including ready, serving and terminating. Eligibility is not a universal guarantee that a real request succeeds. Options such as publishNotReadyAddresses can deliberately change how readiness is represented for discovery.
Registration and health are separate responsibilities
Registration tells the catalogue that an instance exists at a location. An application can register itself, an agent can register it, or platform controllers can maintain entries. Consul documents several registration methods; no universal discovery protocol dictates one lifecycle.
Health monitoring can influence which entries are returned or selected. Active checks send probes; heartbeat or lease mechanisms expect timely updates. A service that cannot renew its lease might have failed, but a network problem between it and the registry can produce the same observation.
A TCP check can confirm that a port accepts connections without proving useful application behaviour.
An HTTP readiness endpoint can test the dependencies needed for normal requests.
A heartbeat confirms recent reporting, not reachability from every client.
Checks need appropriate intervals, timeouts and failure thresholds.
Removing a target from new selections does not rescue an existing failed connection.
Consul’s health check documentation includes passive TTL checks that become critical when status updates stop. In Kubernetes, readiness, liveness and startup probes have different purposes: traffic eligibility, restart decisions and startup tolerance. A liveness check should not restart every instance merely because a shared dependency is briefly unavailable.
DNS caching and stale endpoints
Caching reduces lookup traffic and can improve responsiveness, but it lets clients retain older information. A DNS TTL describes how long a record can be cached; it is not an instruction to move an established connection to another server when that time expires.
Inspect the full chain: registry state, authoritative answer, recursive resolver cache, operating system cache, application resolver and connection pool. A client that resolves once at startup may behave differently from one that follows record lifetimes and reconnects correctly.
Consul’s DNS configuration reference documents the trade between lookup load and freshness, including stale reads and configurable TTLs. A fresh query can still receive older catalogue state if the selected consistency mode permits it.
Negative answers can also be cached. If a lookup runs before a service becomes available, a cached absence may persist after registration succeeds. RFC 2308, updated by later DNS specifications, defines negative caching behaviour. Check the actual resolver and application policies rather than assuming a restart of the service clears all client caches.
Techie Broadband calculation: why failure detection is not instant failover
Consider a deliberately simplified model: a monitor probes every 10 seconds, each failed probe times out after 2 seconds, one failure excludes the endpoint, and the directory updates immediately after that timeout. Clients respect a 30 second DNS TTL, and no other replication or refresh delay exists.
Stage | Assumption | Maximum delay in this model |
|---|---|---|
Wait for next probe | Failure occurs just after a successful check. | Almost 10 seconds. |
Probe timeout | Next probe receives no usable response. | 2 seconds. |
Client cache expiry | Client refreshes just before removal and retains that answer. | Almost 30 more seconds. |
Combined stale selection window | Sequential detection and cache delays. | Almost 42 seconds. |
Techie Broadband calculation: 10 + 2 + 30 = 42 seconds. This is an illustrative upper window for those assumptions, not a measured product result or a universal outage duration. A client can fail earlier, try another endpoint or refresh sooner. Several required failed probes, replication delay, application caching or a long connection timeout can make the real situation different.
If the same model uses a 5 second TTL, the corresponding window becomes almost 17 seconds. That improves freshness but increases lookup demand. For 1,000 clients each querying immediately after every TTL expiry, 30 second caching averages about 33.3 queries per second; 5 second caching averages 200, six times as many. Those figures exclude shared caches, other queries, bursts and connection reuse.
The useful engineering question is therefore how long clients can continue selecting stale endpoints, not simply whether the catalogue contains the correct address now.
Registry reliability, retries and security
The discovery system is itself a dependency. A registry outage may leave established connections working while new or restarted clients cannot obtain endpoints. Cached answers can preserve availability but may select removed instances. A network partition can make freshness and continued operation competing priorities.
Protect registration permissions so unauthorised systems cannot replace legitimate endpoints.
Authenticate and encrypt registry communication where required.
Verify the service identity with appropriate TLS and application authentication after discovery.
Restrict who can browse sensitive internal service information.
Use bounded retries with backoff and jitter rather than having every client retry continuously.
Plan draining and deregistration for maintenance, not only detection of unexpected failure.
Discovery metadata is not permission to access a service. DNSSEC can authenticate appropriately signed DNS data, but it does not grant access to an application or prove that application is functioning. A network name is also not a substitute for certificate hostname validation.
What UK broadband changes, and what it does not
Finding a printer on your local link or a database inside a local home lab normally does not depend on FTTP or FTTC capacity. If local discovery fails, check advertising, multicast, permissions, VLANs and host configuration before blaming the ISP.
Remote discovery is different. A registry, cloud DNS service or VPN reachable across broadband depends on that network path. Outages and latency can delay lookups or lease updates, but a faster package cannot correct the wrong namespace, missing registration or a blocked port.
Publishing a home service name does not create inbound access through CGNAT. Public IPv4, usable IPv6, a VPN or an appropriately designed tunnel must provide the relevant path. Keep internal catalogue entries separate from public hosting arrangements.
Once clients connect, bulk data is limited by the actual application path. For a remotely accessed home NAS, home upload can limit the remote download. Upload speed, router processing, storage, WiFi and the remote destination still matter. Discovery reduces dependence on fixed endpoint configuration; it does not add bandwidth.
Which discovery method fits your setup?
Situation | Useful starting point | Check before adding complexity |
|---|---|---|
Small, stable home service | Local DNS name or router reservation. | Is automatic service browsing actually required? |
Containers in one Compose project | Compose service names on the shared network. | Correct network, container port and reconnect behaviour. |
Kubernetes applications | Services, cluster DNS and EndpointSlices. | Namespace, selector, readiness and network policy. |
Dynamic services across VMs or platforms | A maintained registry with DNS or API access. | Registration ownership, health, consistency and recovery. |
Printers and supported local devices | DNS-SD with the discovery mechanism the product supports. | Local link, permissions and segment boundaries. |
Use existing platform capabilities where they meet the requirement. AWS Cloud Map, for example, supports discovery through configured namespaces using DNS or API calls. That is a product option, not a requirement for every application. A single stable service does not automatically justify another distributed system.
How to troubleshoot service discovery
Identify the discovery mechanism and scope: local mDNS, a Docker network, Kubernetes DNS, unicast DNS or a registry API.
Check publication or registration. Confirm the exact service name, namespace, address family, port and relevant metadata.
Query from the same host, container, Pod or network segment as the failing client. A successful lookup from your laptop may test a different resolver and route.
Separate no answer, an old answer and a valid but unreachable answer. Inspect positive and negative cache behaviour.
Check endpoint eligibility and health data. Confirm that the actual query filters for the state you expect.
Test the selected endpoint and application protocol from the client’s network context, preserving required TLS and HTTP hostnames.
Inspect routing, firewalls, VLAN policy and platform network rules. Then check application authentication and certificates.
Compare lookup timestamps, registry updates and connection errors during one controlled restart or endpoint change.
Review retries, connection pools and resolver behaviour before changing several layers at once.
Kubernetes’ Debug Services guide follows the same separation between DNS, Service configuration, endpoints and functioning Pods. Ping alone does not test the application port or the correctness of a discovery record.
Kubernetes inspection example
These commands inspect a Service named inventory in namespace lab. They require kubectl, suitable cluster access and an existing Service. They are examples, not commands run against a measured Techie Broadband cluster.
kubectl get service inventory -n lab -o yaml
kubectl get endpointslices -n lab -l kubernetes.io/service-name=inventory -o yaml
Compare the selector, ports and endpoint addresses with the intended Pods. An existing Service with no expected endpoints suggests a different problem from a DNS query that never reaches the cluster resolver.
Bonjour browsing example on macOS
Apple’s Bonjour getting started guide demonstrates browsing advertised SSH services using the dns-sd command:
dns-sd -B _ssh._tcp local.
This lists matching advertisements in the local domain; it does not enable SSH or prove login will work. No results can mean that no matching service is advertised. Stop the ongoing browse with Control+C. Other operating systems need an appropriate discovery tool and permission model.
Common service discovery misconceptions
A resolved name proves only that the lookup returned information, not that the application works.
Automatic registration does not necessarily mean active health checking.
A short DNS TTL does not force existing connections to move immediately.
A Kubernetes Service name is not automatically a public hostname.
mDNS discovery across a local link is not the same as internet DNS.
A discovery reflector does not replace application firewall rules.
More broadband bandwidth does not repair stale registrations or incorrect service names.
Frequently asked questions
Is service discovery the same as DNS?
No. DNS can provide a discovery interface, while registries and platform APIs can provide other interfaces. DNS-SD is a particular service browsing and resolution convention. The wider concept includes how endpoint information is published, maintained and consumed.
Does service discovery require a central registry?
No. Local multicast discovery can use advertisements from devices themselves. Other designs use DNS infrastructure, a service catalogue or platform state. Choose the mechanism that fits the scope and lifecycle.
Can it work without internet access?
Yes, when the required discovery and application systems are local and available. Local mDNS and an entirely local container environment can operate without broadband. External registries and remote services introduce separate connectivity dependencies.
Does it automatically make services highly available?
No. It can help clients find replacement or eligible instances, but those instances need capacity and working dependencies. Detection, update propagation, caches and client retry behaviour determine how failure affects users.
Why does a service appear but refuse the connection?
Its advertised address may be wrong or stale, the application may not be listening, or a firewall may block the port. Certificates, authentication and application readiness are additional checks after discovery succeeds.
Key takeaway
Service discovery keeps logical service names connected to changing network endpoints. Understand who publishes the information, who selects an instance and how updates reach clients. Then test reachability, identity and application behaviour separately. That separation makes both home lab troubleshooting and distributed application design much clearer.
To explore the related technical concepts used in this Service Discovery guide, visit our broadband terminology guide.
